Mailman 3 February 2021 - Python-ideas

Multi-line lambda as last argument
by Aleksey Ploskov Feb. 23, 2021

Feb. 23, 2021

Hello, Multi-line lambda is element of many languages, but they are not needed in the form in which they are made in many languages. А good use would be to use the lambda as the last argument of the function. Syntax: > def func(lambda1): > lambda1() > > func(): > print(1) > Can be used for create dsl. For example html: > html: > head() > > body(): > p(): > "Hello world" > a(href="python.org"): > "Python link" > Or for create new syntax constructions: > def unless(condition, _lambda): > if not(condition): > _lambda() > > unless(0): > print('yes') >

1 0

Re: Inadequate error reporting during function call setup stage
by Peter Otten Feb. 23, 2021

Feb. 23, 2021

On 21/02/2021 23:06, Terry Reedy wrote: > On 2/21/2021 12:04 PM, Paul Sokolovsky wrote: > >> Traceback (most recent call last): >> File "pseudoc_tool.py", line 91, in <module> >> first_class_function_value(func, **pass_params) >> TypeError: print() got an unexpected keyword argument 'noann' > > This is not typical behavior in current Python (3.8+). The way I understand it's not about print(), it's about disambiguating multiple functions with the same name. Example: PS > type .\ambiguous_names.py import random def do_stuff(): pass f = do_stuff def do_stuff(a, b): pass g = do_stuff random.choice([f, g])(42) PS > py .\ambiguous_names.py Traceback (most recent call last): File "...\ambiguous_names.py", line 13, in <module> random.choice([f, g])(42) TypeError: do_stuff() missing 1 required positional argument: 'b' The traceback gives no clue which of the two do_stuff() functions caused the error, you have to check both implementations. If that is a comman problem one might consider including module name and co_firstlineno in the message, or at least adding the relevant do_stuff() function to the exception's args.

6 18

Inadequate error reporting during function call setup stage
by Paul Sokolovsky Feb. 21, 2021

Feb. 21, 2021

Hello, Here's example: Traceback (most recent call last): File "pseudoc_tool.py", line 91, in <module> first_class_function_value(func, **pass_params) TypeError: print() got an unexpected keyword argument 'noann' Ok, which "print" do you mean, dear CPython? I have a dozen of print functions (mostly methods) around. So I now need to reduce to printf debugging to find which one is actually meant. And what if that happens in 3rd-party library installed in /usr/lib? On a production system? The most interesting fact is that despite functions has always been first-class values and 30-years language history, such issues are still there. I'm fixing this right away in my Pycopy dialect, because while it's minimalist and I tremble over each half-byte added, but it's Python first of all, and Python means good error reporting. Well, actually I don't fix it right away, but work around, put putting source loc in TypeError's error message. And here's an idea of what the real fix should be: The fact that CPython (and Pycopy) sets up function's frame "outside" the function execution is an implementation detail. The model of execution should be: there's a CALL_FUNCTION/CALL_FUNCTION_KW opcode, and after it executes, the context switches to the target function, and that's *when* argument checking happens. That would give the desired behavior that the last entry in traceback is the actual function which was called and which failed parameter checks. Again, it's an implementation detail if some project doesn't do it like that, and performs param checking before actually pointing VM at the new bytecode function (from which location traceback will be automatically constructed). If it does that, it just still need to append an additional traceback entry pointing to the beginning of the target function. I now implemented that too, and now everything makes sense: Traceback (most recent call last): File "pseudoc_tool.py", line 91, in <module> File "../xforms.py", line 25, in print TypeError: unexpected keyword argument 'noann' -- Best regards, Paul mailto:pmiscml@gmail.com

1 0

Syntax for duplicate types in seq[...]
by wyz23x2＠163.com Feb. 20, 2021

Feb. 20, 2021

Imagine specifying multiple same types. Awful. ``` from typing import Tuple def time() -> Tuple[int, int, int, int, int, int, int, str]: """ Return (year, month, day, hour, minute, second, millisecond, timezone name). ... ``` So I suggest a new syntax, ``Seq[type * times]``, to solve this problem: ``` from typing import Tuple def time() -> Tuple[int * 7, str]: ... ```

3 2

Conditional with statements
by Jonathan Crall Feb. 16, 2021

Feb. 16, 2021

I'm not sure if this has been asked / suggested before. I'm wondering if there is any interest in conditional or loop-based `with` statements. I think it could be done without a syntax change. ### Napkin proposal Context managers could define `__if__` or `__for__`, and if those dunder methods were defined, then the `with` statement would either behave like a conditional or a loop. If `__if__` was defined then ``` with Ctx(): print('hi') ``` would only print `hi` if `__if__` returned True. This doesn't require a syntax change. The `__for__` variant would likely need a minor syntax change. ``` with item in Ctx(): print(item) ``` The `__for__` method is a generator that generates arguments of a loop. The item will be printed as many times as there are items generated by `__for__`. ### Use Cases This would simplify usage of my Timerit and ubelt library. The timerit module defines ``timerit.Timerit``, which is an object that is iterable. It has an ``__iter__`` method that generates ``timerit.TimerTimer`` objects, which are context managers. >>> import math >>> from timerit import Timerit >>> for timer in Timerit(num=200, verbose=2): >>> with timer: >>> math.factorial(10000) The timer context manager measures how much time the body of it takes by "tic"-ing ``__enter__`` and "toc"-ing on ``__exit__``. The underlying object has access to the context manager, so it is able to read its measurement. These measurements are stored and then we compute some statistics on them. Notably the minimum, mean, and standard-deviation of grouped (batched) running times. Unfortunately the syntax is one line and one indent bulker than I would prefer. However, a more consice version of the synax is available. >>> import math >>> from timerit import Timerit >>> for _ in Timerit(num=200, verbose=2): >>> math.factorial(10000) In this case the measurement is made in the `__iter__` method ``Timerit`` object itself, which I believe contains slightly more overhead than the with-statement version. (I should test to determine if this is the case). In the case where it does make a difference, a cool syntax might look like: >>> import math >>> from timerit import Timerit >>> with timer in Timerit(num=200, verbose=2): >>> math.factorial(10000) The other case is that my ``ubelt.Cacher`` library. Currently it requires 4 lines of boilerplate syntax. >>> import ubelt as ub >>> # Defines a cache name and dependencies, note the use of `ub.hash_data`. >>> cacher = ub.Cacher('name', cfgstr=ub.hash_data('dependencies')) # boilerplate:1 >>> # Calling tryload will return your data on a hit and None on a miss >>> data = cacher.tryload() # boilerplate:2 >>> # Check if you need to recompute your data >>> if data is None: # boilerplate:3 >>> # Your code to recompute data goes here (this is not boilerplate). >>> data = 'mydata' >>> # Cache the computation result (pickle is used by default) >>> cacher.save(data) # boilerplate:4 But a conditional ``with`` syntax would reduce boilerplate to 3 lines. >>> import ubelt as ub >>> with ub.Cacher('name', cfgstr=ub.hash_data('dependencies')) as cacher: >>> data = 'mydata' >>> cacher.save(data) >>> data = cacher.data I'm sure there are a lot of viable syntax variations, but does the idea of a conditional or loop aware "with" statement seem like a reasonable language proposal? -- -Dr. Jon Crall (him)

13 33

Adding PathLiteral class and existing object instance to pathlib
by mwmajewsk Feb. 16, 2021

Feb. 16, 2021

Hi All, The pathlib library is really helpful when working with paths, but creating an instance of Path class could be easier. There was a discussion about solving this with p-strings here https://mail.python.org/archives/list/python-ideas@python.org/thread/MNKRQF… But that would alter python syntax. Easier, but just as nice solution could be adding the following snippet to pathlib: class PathLiteral(): def __truediv__(self, other): return Path(other) path_literal = PathLiteral() This would allow the following snippet to work: >>> from pathlib import path_literal as p >>> p/"foo" PosixPath('foo') This idea is inspired by this tweet https://twitter.com/bitecode_dev/status/1217429169396731906 The core of the idea is that it does not require syntax changes, only need a small instance of an object, that can simplify and encourage use of pathlib. When someone needs to rewrite a lots of string-paths to pathlib's Path you only need to add that import line, and prepend strings with "p/". Cheers, MM

3 4

Pattern-Matching for functions
by Abdulla Al Kathiri Feb. 14, 2021

Feb. 14, 2021

In some functional programming languages, they can pattern match a function based on its parameters. Similar to function overloading. Will Python Pattern Matching do something similar to functions? If calling a function doesn't match any of the cases and catch-all case is not present, then some sort of new or exisiting Exception needs to be raised. Type annotations can be used (for the parameters, they may be redundant, but for the return, they can be useful). The Return annotation comes right after the guard. See theortical example below. class Point: def __init__(self, pos): self.pos = pos def write(self, msg): return f"writing {msg}" match def f: case (float(x) | int(x), float(y) | int(y)) if x > 0 -> int: return round(x + y) # f(3.4, 7) => 10 case (int(x) | float(x)): return round(2 * x) # f(8) => 16 case (0): return 0 # f(0) => 0 case (0, 0): return "Origin" # f(0, 0) => "Origin" case ([int(x) | float(x), int(y) | float(y)], int(z) | float(z)): return x + y + z # f((5, 2.5), 2.5) => 10 case ( (Point(pos=(int(x), int(y)))) as point, *_, message=str(msg), **_, ) if x > 0 and y > 0: txt = point.write(msg) res = x + y return res, txt # point = Point([1, 9]) # f(point, message="Hello") => 10, "writing Hello" case (*_, **_): return "Catch_All" ## which can be shortened to... case _: return "Catch_All" #f(1, 3, 4, 5) => “Catch_All"

1 0

Pattern Matching -- Removing Name Binding from Partial Matching
by Abdulla Al Kathiri Feb. 14, 2021

Feb. 14, 2021

I will explain it in the following few lines of code.. name = "George" year = 2021 d = {"name": "Mike", "year": 1919} match d: case {"name": name, "year": 1917}: print("match 1 found”) # I want to remove binding "name" here from partial matching case {"year": year, "name": "Michael”}: print("match 2 found”) # I want to remove binding "year" here from partial matching. # Basically removing all name bindings after every partial/failed matching case _: print("match not found!”) print(f"{name = }, {year = }”) ### Output ###: match not found! name = 'Mike', year = 1919 But I want :var: 'name' to stay being the global name “George" and :var: 'year' being the global year 2021 if an exact matching is not found. Maybe it is done the way it is for speed optimization (overhead reduction or something), but what if I don't care about speed and I care about having no un-intentional side-effects? Can we do something like the following to solve this issue? match d, partial_binding=False: case … : ... case … : ... with partial_binding=True by default. Any other idea in how to prevent name binding due to partial matching from happening? Any previous discussions on this? Thanks, Abdulla

3 5

Dead simple import hooks - proposed solution
by Paul Sokolovsky Feb. 13, 2021

Feb. 13, 2021

Hello, Some time ago I posted a query regarding how the simplest possible (while still practical to use) import hooking might look like. Since then, I've coded up my try: https://github.com/pfalcon/python-imphook . I'd be interested in hearing any criticism, in particular in regard to the claims of minimalism and being "dead simple". Below is the copy of README for your reference (documentation is still WIP). A few initial examples are available in the git repo above. imphook - Simple and clear import hooks for Python ================================================== The ``imphook`` module allows to easily define per file type import hooks, i.e. overload or extend import processing for particular file types, without affecting processing of other file types, and at the same time, while ensuring that new processing integrates as seamlessly as possible with normal Python import rules. Besides the Python-level API to install import hooks, the module also provides command-line interface to run an existing Python script or module with one or more import hooks preloaded (i.e. without modifying existing script source code). Some but not all things you can easily do using ``imphook`` (most of these require additional modules to do the heavy lifting, ``imphook`` just allows to plug it seamlessly into the Python import system): * Override importing of (all or some) .py files, to support new syntax or semantics in them. * Import files written using a DSL (domain-specific language) as if they were Python modules. E.g., config or data files. * Import modules written in other language(s), assuming you have an interpreter(s) for them. * Import binary files, e.g. Java or LLVM bytecode. ``imphook`` works both with new, lightweight legacy-free Python API, as promoted by the `Pycopy <https://github.com/pfalcon/pycopy>`_ Python dialect (the original source of the "easy import hooks" idea), and CPython (the older, reference Python implementation), and with other Python implementations which are CPython-compatible. Quick Start ----------- Make sure that you already installed ``imphook`` using:: pip3 install -U imphook Below is a complete example of an import hook module to load ``key = value`` style config files:: import imphook def hook(modname, filename): with open(filename) as f: # Create a module object which will be the result of import. mod = type(imphook)(modname) for l in f: k, v = [x.strip() for x in l.split("=", 1)] setattr(mod, k, v) return mod imphook.add_import_hook(hook, (".conf",)) Save this as the ``mod_conf.py`` file, and add the two following files to test it: ``example_settings.conf``:: var1 = 123 var2 = hello ``example_conf.py``:: import example_settings as settings print(settings.var1) print(settings.var2) Now run:: python3 -m imphook -i mod_conf example_conf.py As you can see, the ``example_conf.py`` is able to import ``example_settings.conf`` as if it were a normal Python module. Besides copy-pasting the above and other examples, you can also clone the Git repository of ``imphook``, which contains various ready-to-use examples:: git clone https://github.com/pfalcon/python-imphook API to install hooks and hook structure --------------------------------------- The API of the module consists of one function: `imphook.add_import_hook(hook, ext_tuple)`. *hook* is a name of hook function. *ext_tuple* is a tuple of file extensions the hook function should handle (the leading dot should be included). More often than not, you will want to handle just one extension, so don't forget to use the usual Python syntax with a trailing comma for 1-element tuple, e.g.: ``(".ext",)``. Python modules may not contain a dot (``"."``) in their names (they are used to separate subpackages), so the extension you register may contain multiple dots, e.g. ``".foo.bar"``, with filename ``my_module.foo.bar`` matching it. It is possible to call `imphook.add_import_hook(hook, ext_tuple)` multiple times to install multiple hooks. The hooks are installed in the stack-like fashion, the last installed will be called first. It is possible to install multiple hooks for the same file extension, and earlier installed hooks may still be called in this case, because a hook function may skip processing a particular file, and let other hooks to take a chance, with default processing happening if no hook handled the import. The signature and template of the actual hook function is:: def my_hook(modname, filename): # Return None if you don't want to handle `filename`. # Otherwise, load `filename`, create a Python module object, # with name `modname`, populate it based on the loaded file # contents, and return it. The *modname* parameter is a full module name of the module to import, in the usual dot-separated notation, e.g. ``my_module`` or ``pkg.subp.mod``. For relative imports originated from within a package, this name is already resolved to full absolute name. The *modname* should be used to create a module object with the given name. The *filename* parameter is a full pathname (with extension) of the file which hook should import. This filename is known to exist, so you may proceed to open it directly. You may skip processing this file by returning ``None`` from the hook, then other hooks may be tried, and default processing happens otherwise (e.g. ``.py`` files are loaded as usual, or ImportError raised for non-standard extensions). For package imports, the value of *filename* ends with ``/__init__.py``, and that is the way to distinguish module vs package imports. If the hook proceeds with the file, it should load it by whatever means suitable for the file type. File types which are not natively supported by Python would require installing and using other extension modules (beyond ``imphook``). After loading the file, the hook should create an empty Python module object which will be the result of the import. There are a few ways to do that: * The baseline is to call a module type as a constructor. To get a module type, just apply the usual ``type()`` function to an existing (imported) module. You'll definitely have ``imphook`` itself imported, which leads us to:: mod = type(imphook)(modname) The parameter to constructor is the name of module to create, as passed to the hook. * If the above looks too magic for you, you can import symbolic name for module type from the ``types`` module:: from types import ModuleType mod = ModuleType(modname) * Finally, you may use the ``imp`` module, which may be as well the clearest (to the newbie) way of doing it:: import imp mod = imp.new_module(modname) But mind that the ``imp`` module is considered deprecated. Of the choices above, the first is the most efficient - no need to import additional modules, and it's just one line. And once you saw and were explained what it does, it shouldn't be a problem to remember and recognize it later. Once the module object is created as discussed above, you should populate it. A way to do that is by using ``setattr()`` builtin to set a particular attribute of a module to a particular value. Attributes usually represent variables with data values, but may be also functions and classes. Finally, you just return the populated module object. In case you want to perform custom transformation on the Python source, the process is usually somewhat different, where you transform a representation of the source, and then execute it in the context of a new module, which causes it to be populated. -- Best regards, Paul mailto:pmiscml@gmail.com

1 0

Gauging appetite for a divergent alternative to pattern matching
by Ambient Nuance Feb. 11, 2021

Feb. 11, 2021

Pattern matching re-surfaced on my radar today, along with the impression of a conclusion being just around the corner. Although this has seemed an ongoing state at times, a certain finality would come from a steering council decision. At the same time, it seems the polarisation that has developed around this feature has solidified more than dissipated. So I am prompted to throw my hat in the ring of language design. However, I must acknowledge that it is very late in the development cycle of this feature. Given the sheer volume of previous discussion, I am uncertain if there is an appetite for further deliberation at this point. This post is an invitation to help clarify such uncertainty. What I can bring to the table is a syntax design that is structurally close to PEP 634, but diverges significantly in detail and priorities. As to the content of this design, it seems best for the moment to give only a brief outline of significant differences. Laying out specific syntactic proposals seems a sure way to ignite a discussion which some/many may not wish to have. - on a general level, it is a superset of both pattern matching and type hints. - maybe this interferes with static typing efforts? Maybe it's a more flexible common ground? - prioritises consistency with current syntax and 'style' of Python. - to do so, introduces a new dunder method (single argument, boolean return). - design began from interrogating arbitrary Python objects, rather than tuple unpacking. - more 'object' oriented, but requires explicit/verbose notation of 'store' variables. - centres around a new 'template' class, which any object can be 'queried' against. - these can be created, customised and re-used outside of 'match' statements. Since I didn't set myself much of a conceptual ceiling, several optional extensions came about. - a guard template to control whether the 'match' block is entered (e.g. integers only, then different ranges). - opt-in fall-through on a per-branch basis. - an additional dunder method to improve brevity of repeated templates. - a new operator to utilise the new dunder method outside of 'match' contexts. - a separate but related 'assign' block statement (shortcut for assignment to a common variable). - outside of a 'match' context, using multiple template dispatch for functions/methods. Whatever the outcomes of this post and PEP 643, I will continue with my development of prototype emulations for current Python syntax. Discussion in this thread will help me decide on the degree of pace and detail to put into coding and communication efforts, e.g. PyPI and blog vs. full-on PEP. Below is a description of my own context in relation to pattern matching. It is included primarily to elaborate on the motivation and timing of this post. --- Shortly after the original publication of PEP 622, I began a loosely-bound train of thought on syntactic sugar in Python. My particular focus was on the clarity and consistency of future possibilities. Cue many iterations and multiple restarts from scratch. I ended up with syntax designs ranging from generalised container merging to graph data structures. In order to more thoroughly iron out shortcomings, I have been on-and-off developing emulations using current Python syntax. Meanwhile, pattern matching has continued on its own steam in the tuple-unpacking direction. After reading a variety of discussions, my takeaway impression was that in most cases the design was not particularly open to substantive changes. This seemed true at both a minute and broad scale. So I stepped away from these discussions to take the time to flesh out my own thoughts. For myself, the most exciting idea within PEP 622 is switch-based control flow driven by custom predicates. In the current proposal, such predicates are optimised for a combination of structural unpacking and class inheritance. A kind of inline abbreviated-expression dispatch. The escape hatch that I found to diversify predicate-space was to delegate all 'matching' details to a new class of objects. This enables the control of predicate logic outside of a 'match' statement, sidestepping the need to compress such details into an inline format. On a final note... As others do, I have my own likes and dislikes about the current form of pattern matching. Regardless of these opinions, I want to make clear that it is not my intention with this post to make a divisive affront to previous and ongoing efforts. If others wish to construe it as such, or take it to such a place, please instead spend your efforts building a reflective and much-less-binary dialogue around language design, rather than language designers.

3 2