Mailman 3 May 2021 - Python-ideas

SEMANTICS for 'variables' produced by a decorator
by Jonathan Fine May 29, 2021

May 29, 2021

There's been a discussion in this list on extending Python to provide SYNTAX such as @decorator name = EXPRESSION and also suitable semantics. (Here 'name' is an identifier, in the discussion called a 'variable'.) This post is about providing SEMANTICS for such decorator syntax. We can do this within the existing Python syntax for decorators. Recall that @decorator def fn(....): # body is usually equivalent to def fn(....): # body fn = decorator(fn) and that decorator is just a callable that returns something. It need not return another function. It could return a 'variable', such as the result of calling fn. Here's a proof of concept. Consider the following BEGIN $ cat work.py from collections import namedtuple Locn = namedtuple('Locn', 'doc module name') def locn_from_fn(fn): name = fn.__name__ module = fn.__module__ doc = fn.__doc__ return Locn(name=name, module=module, doc=doc) def decovar(fn): locn = locn_from_fn(fn) return fn(locn) @decovar def variable(locn): return ('value', locn) def deconamedtuple(fn): locn = locn_from_fn(fn) nt = namedtuple(locn.name, fn()) nt.__doc__ = locn.doc nt.__module__ = locn.module nt.__name__ = locn.name return nt @deconamedtuple def Point(): '''Return a point (x, y) in the plane.''' return 'x y' print(variable) print(Point) print(Point.__doc__) END Here's what we get when we run the script. BEGIN $ python3 work.py ('value', Locn(doc=None, module='__main__', name='variable')) <class '__main__.Point'> Return a point (x, y) in the plane. END It should now be clear that this approach allows us to pass much useful information to the decorator. I claim that this approach gives most or all of the semantic benefits of 'decorators on variables', and within the current Python syntax. If so, and the semantic benefits are strong, then here's part of a good case for extending the syntax in a future version of Python. -- Jonathan

1 0

Introduce constants in Python (constant name binding)
by Shreyan Avigyan May 29, 2021

May 29, 2021

I posted my previous idea regarding this on the mailing list. This idea is a little different. This idea suggests introducing constant name bindings. This is similar to const pointer in C/C++. Once a name has been assigned to a data we can change the data (if mutable) but we cannot change the name to point to a different data. The only way the data the constant points can get deallocated is if it goes out of scope, the program exits or the constant is manually `del` by the user. The proposed syntax is as follows, constant x = 10 constant y = ["List"] constant z: str = "Hi" Thanking you, With Regards

17 68

Add static variable storage in functions
by Shreyan Avigyan May 29, 2021

May 29, 2021

Lot of programming languages have something known as static variable storage in *functions* not *classes*. Static variable storage means a variable limited to a function yet the data it points to persists until the end of the program. Well Python also kind of has that functionality. Python's default values provide the same type of functionality but it's a *hack* and also *problematic* because only mutable types that are mutated persists. Static should behave much like Python's for loop variables. This idea proposes to add a keyword (static, maybe?) that can create static variables that can persist throughout the program yet only accessible through the function they are declared and initialized in. Thanking you, With Regards

11 65

Re: Introduce constants in Python (constant name binding)
by Christopher Barker May 28, 2021

May 28, 2021

oops, forgot to include the list. i really hate lists that don't have the the list as the default reply setting :-( (Yes, I know that goes against the conventional wisdom) On Wed, May 26, 2021 at 9:14 AM Shreyan Avigyan <pythonshreyan09(a)gmail.com> > wrote: > >> Well. How can I go beyond why constant was invented in the first place? > > > But constant was invented for other, far more static languages. And many > in this thread have said that even then it often needs to be worked around. > > > The main reason constant was invented was to provide an additional >> support to programmers so that they don't make a program unstable. > > > The same could be said for static typing. > > And Python’s solution to that is type annotations and type checkers. Which > supports Final. > > In short, yes, the concept of constant is useful, but it is not a good fit > for Python. > > However: > > When you spoke of “like a literal” that brought to mind something else: a > C preprocccesor like pre-complier time substitution. Which might have some > use, but it would only work within a single module, so not very useful. > > -CHB > > > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

2 1

name for new Enum decorator
by Ethan Furman May 28, 2021

May 28, 2021

Greetings! The Flag type in the enum module has had some improvements, but I find it necessary to move one of those improvements into a decorator instead, and I'm having a hard time thinking up a name. What is the behavior? Well, a name in a flag type can be either canonical (it represents one thing), or aliased (it represents two or more things). To use Color as an example: class Color(Flag): RED = 1 # 0001 GREEN = 2 # 0010 BLUE = 4 # 0100 PURPLE = RED | BLUE # 0101 WHITE = RED | GREEN | BLUE # 0111 The flags RED, GREEN, and BLUE are all canonical, while PURPLE and WHITE are aliases for certain flag combinations. But what if we have something like: class Color(Flag): RED = 1 # 0001 BLUE = 4 # 0100 WHITE = 7 # 0111 As you see, WHITE is an "alias" for a value that does not exist in the Flag (0010, or 2). That seems like it's probably an error. But what about this? class FlagWithMasks(IntFlag): DEFAULT = 0x0 FIRST_MASK = 0xF FIRST_ROUND = 0x0 FIRST_CEIL = 0x1 FIRST_TRUNC = 0x2 SECOND_MASK = 0xF0 SECOND_RECALC = 0x00 SECOND_NO_RECALC = 0x10 THIRD_MASK = 0xF00 THIRD_DISCARD = 0x000 THIRD_KEEP = 0x100 Here we have three flags (FIRST_MASK, SECOND_MASK, THIRD_MASK) that are aliasing values that don't exist, but it seems intentional and not an error. So, like the enum.unique decorator that can be used when duplicate names should be an error, I'm adding a new decorator to verify that a Flag has no missing aliased values that can be used when the programmer thinks it's appropriate... but I have no idea what to call it. Any nominations? -- ~Ethan~

1 0

division of integers should result in fractions not floats
by Martin Teichmann May 27, 2021

May 27, 2021

Hi list, when dividing two integers, the result is a float, which means we immediately lose precision. This is not good if you want to use code which supports higher precision. Decimals come to mind, but also sympy. This loss of precision could be avoided if the result of a division is a fraction instead: a fraction is exact. So when writing Decimal(1/3), currently we lose the precision in the division, something that the Decimal module cannot undo. With my proposal, the entire precision is retained, and it works as expected. This is even more clear for sympy, a Package for symbolic calculations: currently, sympy cannot do much about "1/2 * m * v**2", although it looks like a perfectly fine formula. But sympy only sees "0.5" instead of "1/2", which is not usable in symbolic calculations. I am aware that this would be a huge change. But we have had such a change in the past, from integers having a floor division in Python 2, to a float in Python 3. Compared to this, this is actually a small change: the value of the result is only different by the small pecision loss of floats. The bigger problem is caused by the fact that some code may rely on the fact that a value is a float. This can be fixed easily by simply calling float(), which is also backwards-compatible, it will work on older versions of Python as well. I have prototyped this here: https://github.com/tecki/cpython/tree/int-divide-fraction The prototype uses the fractions.Fraction class written in Python as result for integer true divisions. I expected that to go horribly wrong, but astonishingly it did not. Only a small number of tests of Python fail, mostly those where it is explicitly tested whether an object is a float. So I lowered the bar even more and tried to compile and test numpy. And also there, except some tests that very explicitly require floats, it worked fine. In order to showcase how that would look like, let me give an example session: >>> 5/6-4/15 17/30 >>> a=22/7 >>> f"{a}" '22/7' >>> f"{a:f}" '3.142857' >>> from decimal import Decimal >>> Decimal(1/3) Decimal('0.3333333333333333333333333333') As a comparison, the same with current Python: >>> 5/6-4/15 0.5666666666666667 >>> a=22/7 >>> f"{a}" '3.142857142857143' >>> f"{a:f}" '3.142857' >>> from decimal import Decimal >>> Decimal(1/3) Decimal('0.333333333333333314829616256247390992939472198486328125') Cheers Martin

24 84

Decorators on variables
by micro codery May 26, 2021

May 26, 2021

Variable decorators have been suggested here before, as have new statements that could also achieve the same level of access to the binding name. However I propose a much more restricted syntax that would make for less edge cases where what is actually passed to the decorator callable may be ambiguous. Basically this would add syntax to python that would transform @decorator("spam this") variable into variable = decorator("variable", "spam this") The decorator would be limited to being on the same line as the variable name. Additionally, the variable can not also be assigned to on the same line, so @decorator("spam this") variable = eggs would be a SyntaxError. Annotating these variable would be allowed @decoratopr("spam this") variable: Food The actual decorator would follow the same rules as current decorators in that the parentheses can be omitted if no other arguments are required to the function and any arbitrary expression can be used, however the same restraint should be cautioned as is for current decorators post PEP 614. The most straightforward use of this decorating would be that str() could be used without change to declare variables that are strings of the same name, generally cutting down on repeating yourself when declaring constants. #instead of GREEN = "GREEN" @str GREEN The classic example of a factory that needs its target name as a string value is namedtuple. I don't think this example is enough alone to sway this list but I bring it up as it has also been at the front of past ideas to allow variable decorating. #instead of Point = namedtuple("Point", "x y z") @namedtuple("x y z") Point I have sometimes heard that dataclasses make this no longer an issue, implying that dataclasses can be used in place of namedtuple in any new code. Putting aside that dataclasses are not a strict replacement for namedtuple, the dataclasses module continues this factory design with the make_dataclass function. #instead of Point = make_dataclass("Point", [("x", int), ("y", int), ("z", int)]) @make_dataclass([("x", int), ("y", int), ("z", int)]) Point Enum is one more case where there is a factory equivalent to the class declaration. #instead of Colors = Enum("Colors", "RED GREEN BLUE") @Enum("RED GREEN BLUE") Colors If you want each enum member value to match its name there is even more repetition, however by using the original example for decorated variables, this can be reduced when using the more common class style. class Colors(Enum): @str RED @str GREEN @str BLUE One final standard library module that I will mention because it seems to keep adding similar factory functions is typing. It already has the factory functions NewType, TypeVar, ParamSpec, TypedDict, NamedTuple. Their use with the new variable decorator would look much the same as earlier examples, but here would be one case where type hinting the variable in such a statement could aid the decorator function #instead of UUIDType = NewType("UUIDType", str) @NewType UUIDType: str If adopted, no change to these functions would be necessary to allow their use as a variable decorators as they all already take the name as a string in the first argument. Additionally, as variable decorators are actually callables just like the current decorators, no new restrictions are being forced on users of the language. If a user really wants to create a namedtuple with a typename of Foo but assign the result to a variable named ___trippleS33cret___name__here_ the language will not stop them. I would very much like to know if any module maintainers are interested in this syntax. Convincing arguments from the standard lib (whether you consider the above convincing or not) are probably not enough to get a syntax change into python. But if enough third party modules want to use this functionality I believe that would be the real push to get it done. Specifically this idea first started growing when I read on another ideas message that using the module sympy would often start with a line like x = symbol('x') which, with no change, could be used as a variable decorator @symbol x Now what may be even more typical (I really have never used the module myself) is defining all needed symbols on one line with x, y, z = symbol('x y z') which could conceivably be rewritten like @symbol x, y, z However I have not yet been able to decide if multi-assignment would be overall beneficial to this new syntax. Probably not, at least at this stage. It would not be hard for a decorator function to return multiple values, but how would the multiple names be passed in? A string with whitespace like this original, and namedtuple's field_names, is possible as no valid identifier can have a space in it. However, it forces any would-be variable decorator to know this and check if a space is in its first argument, even if it cannot produce multiple values. Another option is to pass in an iterable, but the same additional effort would be placed on decorators that otherwise don't want to deal with multi-assignment. Unpacking the names as the first n arguments would mean that decorators that just want one variable can get just that and those that may want more will also get them, but then practically no variable decorator could take additional non-keyword arguments. Regards ~ Jeremiah

8 19

Add support for private variables, methods and functions in Python
by Shreyan Avigyan May 26, 2021

May 26, 2021

Private methods, functions and variables are very common in programming languages. But Python doesn't support private. It has conventions for naming so considered private but not private. Most of the time private is never required, what Python provides is more than enough. But the need for private come into place when we're dealing with passphrases and servers. For example consider this code, class A: def get(): // code to get the password self.password = password Now consider this, >>> x = A(); x.get(); x.password See what just happened? Someone just got the member variable value that the person wasn't supposed to. I suggest to add private support for functions (module __all__ methods to be more clear), methods and variables (module __all__ variables or class variables). (I very bad at reading PEPs so I may miss out something critical that's been explained already (sometimes I miss out a feature in a PEP and think about suggesting that feature when it's already there and then I realize "Oh! It's already here in this PEP"). If that's the case then please correct me.) With Regards, Shreyan Avigyan

15 49

Currying syntax with multiple parameter lists
by Joren Hammudoglu May 25, 2021

May 25, 2021

> Flat is better than nested. I remember being confused a lot when I was learning how to write my first decorator. Now eight years later, with the last two spent full-time writing Python, I introduced a bug in production because I forgot to return the inner function from within a decorator. By introducing multiple parameter lists when defining a function, Scala does a great job at improving readability for curried functions (https://docs.scala-lang.org/tour/multiple-parameter-lists.html). In Python, that could look like e.g.: def log(level=logging.DEBUG, logger=logging.root)(func)(*args, **kwargs): logger.log(level, 'call: %s', func.__qualname__) return func(*args, **kwargs) Which would be sugar for: def log(level=logging.DEBUG, logger=logging.root): def _log(func): def __log(*args, **kwargs): logger.log(level, 'call: %s', func.__qualname__) return func(*args, **kwargs) return __log return _log The obvious problem in this example, is that `functools.wraps` is missing. One solution would be for me to flex my 8 years of experience, and present these two (super/meta-)decorators: def wraps_decorator(decorator: Callable[[F], F]): @functools.wraps(decorator) def _wraps_decorator(func): return functools.wraps(func)(decorator(func)) return _wraps_decorator @wraps_decorator def wraps_decorator_factory(decorator_factory)(*args, **kwargs): return wraps_decorator(decorator_factory(*args, **kwargs)) Applying the latter on the first example, it becomes @wraps_decorator_factory def log(level=logging.DEBUG, logger=logging.root)(func)(*args, **kwargs): logger.log(level, 'call: %s', func.__qualname__) return func(*args, **kwargs) which is equivalent to: def log(level=logging.DEBUG, logger=logging.root): def _log(func): @functools.wraps(func) def __log(*args, **kwargs): logger.log(level, 'call: %s', func.__qualname__) return func(*args, **kwargs) return __log return _log Implementation-wise, I think it's very feasible since it's only sugar. And I'm sure that the required grammar changes are possible with the shiny new parser. And considering PEP 659, I can imagine that this syntax is beneficial because the closure variables coincide with the parameter lists.

2 1

match expressions in python 3.10
by Tyler F May 24, 2021

May 24, 2021

with the addition of PEP 634 and the new match/case syntax in python 3.10, it seems fitting to be able to use these new match statements inside of expressions as well, which is why I think we should have a new `matches` keyword (similar to if/else, is, for, not, etc.) for use in expressions. This makes many more complex if/else statements much less complex (see example use cases at #2), and overall looks much more pythonic in practice. The expression would be higher on the order of operations than the boolean operators, but higher than the comparisons, but this is subject to change. (see #1) However, this should be used sparingly (ideally for single comparisons), similar to list comprehensions versus for loops. 1. GRAMMAR MODIFICATIONS: ``` inversion: | 'not' inversion | match_expression matches_expression: | comparison 'matches' patterns # patterns is defined in PEP 634 | comparison ``` 2. USE CASES: a. ``` if variable matches ["example", *files]: print(files) ``` as opposed to: ``` match variable: case ["example", *files]: print(files) ``` b. ``` for x in some_list: if x matches Class1(attr1=3) | Class2(attr2=4): print("got a match!") break ``` as opposed to ``` for x in some_list: match x: case Class1(attr1=3) | Class2(attr2=4): print("got a match!") break ```

3 4