Mailman 3 October 2021 - Python-ideas

PEP 671 proof-of-concept implementation
by Chris Angelico 30 Oct '21

30 Oct '21

https://github.com/Rosuav/cpython/tree/pep-671 So uhh... anyone who knows about the internals of CPython and wants to join me on this, I would *really* appreciate coauthors! The implementation ended up a lot more invasive than I originally planned. Some of that is inherent to the problem, but other parts might be able to be done more cleanly. The way I've done it: * Argument defaults (either in __defaults__ or __kwdefaults__) are now tuples of (desc, value) or (desc,) for early-bound and late-bound respectively * Early-bound defaults get mapped as normal. Late-bound defaults are left unbound at time of function call. * For each late-bound default as of the 'def' statement, a check is coded: if the local is unbound, set it based on the given expression. This means that it's possible to replace an early-bound default with a late-bound, but instead of actually evaluating the expression, it just leaves it unbound: >>> def f(x=1): print(x) ... >>> f.__defaults__ ((None, 1),) >>> f.__defaults__ = ((None,),) >>> f() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 1, in f UnboundLocalError: cannot access local variable 'x' where it is not associated with a value >>> I'm not entirely happy with this, but I also don't want to have too much of a performance impact in the normal case. So far unimplemented is the description of the argument default. My plan is for early-bound defaults to have None there (as they currently do), but late-bound ones get the source code. (In theory, it may be of value to retain the source code for earlies too, which would allow hex or octal integer literals to show up in help() as such, rather than showing the (decimal) repr of the resulting value.) Anyone got good pointers on how to do this, or is that likely to be impractical? Feel free to criticize my code. As you'll see from the commit messages in that branch, I have no idea what I'm doing here :) ChrisA

6 11

`is in`/`not is in` operators
by Jeremiah Vivian 29 Oct '21

29 Oct '21

I DO expect this thread to be bombarded with negative replies. Currently, there are `in`/`not in` operators which work like this in Python: > def contains(contains_value, iterable, not_in): > for element in iterable: > if element == contains_value: > return True ^ not_in > return False ^ not_in If I wanted to check if an *exact* object is in an iterable, I would have to loop like this (reverse boolean values for implementation of `not is in`): > is_in = False > for element in iterable: > if element is contains_value: > is_in = True I would want a more *convenient* way to check for this value. So therefore, there should be `is in`/`not is in` operators to do it better. Is this a valid reason?

5 20

Python Shared Objects
by byko3y＠gmail.com 28 Oct '21

28 Oct '21

Due to https://www.python.org/dev/peps/pep-0554/ multi-interpreters implementation going really slow, I had the audicity to try an alternative route towards the same objective of implementing multicore support of python: instead of sharing the memory by running multiple threads, I employed an interprocess shared memory with multiple processes. I know there are multiprocessing.sharedctypes and multiprocessing.shared_memory, but I went much deeper into it by solving two problems they failed to solve: sharing of complex dynamic objects and synchronization of data access. I have a working prototype to show: https://github.com/byko3y/python-shared-objects It's a kind of software transactional memory within shared memory. It has a basic support for fundamental python types (bool, int, str, bytes, tuple, list, dict, object), for both atomic and non-atomic transactions via fine-grained RW-locks, has a built-in protection against deadlock and starvation. Most of my code can also be used for cross-interpreter communication after PEP 554 is successfully implemented, since cross-interpreter communication is still an open question.

2 2

Syntax for late-bound arguments
by Chris Angelico 28 Oct '21

28 Oct '21

Spun off from PEP 505 thread on python-dev. One of the weaker use-cases for None-coalescing is function default arguments, where the technical default is None but the function wants to use some other value instead. This is a weak argument in favour of None-coalescing because only a small set of such situations are only slightly improved by a "??" operator, and the problem is far larger. Consider the bisect.bisect() function [1]: def bisect(a, x, lo=0, hi=None): if lo < 0: raise ValueError('lo must be non-negative') if hi is None: hi = len(a) It's clear what value lo gets if you omit it. It's less clear what hi gets. And the situation only gets uglier if None is a valid argument, and a unique sentinel is needed; this standard idiom makes help() rather unhelpful: _missing = object() def spaminate(thing, count=_missing): if count is _missing: count = thing.getdefault() Proposal: Proper syntax and support for late-bound argument defaults. def spaminate(thing, count=:thing.getdefault()): ... def bisect(a, x, lo=0, hi=:len(a)): if lo < 0: raise ValueError('lo must be non-negative') This would be approximately equivalent to the _missing idiom, with the following exceptions: 1) Inspecting the function would reveal the source code for the late-bound value 2) There is no value which can be passed as an argument to achieve the same effect 3) All late-bound defaults would be evaluated before any other part of the function executes (ie the "if hi is None" check would now be done prior to "if lo < 0"). The syntax I've chosen is deliberately subtle, since - in many many cases - it won't make any difference whether the argument is early or late bound, so they should look similar. While it is visually similar to the inline assignment operator, neither form is currently valid in a function's parameter list, and thus should not introduce ambiguity. The expression would be evaluated in the function's context, having available to it everything that the function has. Notably, this is NOT the same as the context of the function definition, but this is only rarely going to be significant (eg class methods where a bare name in an early-bound argument default would come from class scope, but the same bare name would come from local scope if late-bound). The purpose of this change is to have the function header define, as fully as possible, the function's arguments. Burying part of that definition inside the function is arbitrary and unnecessary. ChrisA [1] Technically it's bisect_right but I'm using the simpler name here. Also, it recently grew a key parameter, which would only get in the way here. https://docs.python.org/3/library/bisect.html#bisect.bisect_right

15 53

Parameter tuple unpacking in the age of positional-only arguments
by Erik Demaine 28 Oct '21

28 Oct '21

On Tue, 26 Oct 2021, Eric V. Smith wrote: > You may or may not recall that a big reason for the removal of "tuple > parameter unpacking" in PEP 3113 was that they couldn't be supported by the > inspect module. Quoting that PEP: "Python has very powerful introspection > capabilities. These extend to function signatures. There are no hidden > details as to what a function's call signature is." > > (Aside: I loved tuple parameter unpacking, and used it all the time! I was > sad to see them go, but I agreed with PEP 3113.) Having recently heard a friend say "the removal of tuple parameter unpacking was one thing that Python 3 got wrong", I read this and PEP 3113 with interest. It seems like another approach would be to treat tuple-unpacking parameters as positional-only, now that this is a thing, or perhaps require that they are explicitly positional-only via in PEP 570: def move((x, y), /): ... # could be valid? def move((x, y)): ... # could remain invalid? Is it worth revisiting parameter tuple-unpacking in the age of positional-only arguments? Or is this still a no-go from the perspective of introspection, because it violates "There are no hidden details as to what a function's call signature is."? (This may be a very short-lived thread.) Erik -- Erik Demaine | edemaine(a)mit.edu | http://erikdemaine.org/

3 2

Add __len__ to ipaddress._BaseNetwork
by nyuszika7h＠gmail.com 26 Oct '21

26 Oct '21

Currently, ipaddress._BaseNetwork (and by extension, ipaddress.IPv4Network and ipaddress.IPv6Network) does not have a __len__ method, it only has num_addresses. This makes the following code not work: >>> random.choice(ipaddress.ip_network('6.0.0.0/8')) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/nyuszika7h/.pyenv/versions/3.10.0/lib/python3.10/random.py", line 378, in choice return seq[self._randbelow(len(seq))] TypeError: object of type 'IPv4Network' has no len() The workaround is a bit ugly: >>> (network := ipaddress.ip_network('6.0.0.0/8'))[random.randrange(network.num_addresses)] IPv4Address('6.60.184.142') With a custom subclass, all works well: >>> class MyIPv4Network(ipaddress.IPv4Network): ... def __len__(self): ... return self.num_addresses ... >>> random.choice(MyIPv4Network('6.0.0.0/8')) IPv4Address('6.40.110.63')

2 1

Fwd: Simple curl/wget-like download functionality in urllib (like http offers server)
by Tom P 26 Oct '21

26 Oct '21

In the context of building Docker images, it is often required to download stuff. If curl/wget is available, great, but often slim images don't include that. The urllib could provide a very simple download functionality (like http offers a simple server): from urllib.request import urlopen data = urlopen('https://.../install-poetry.py').read() # print or save data

10 16

Re: PEP 671: Syntax for late-bound function argument defaults
by Jon Kiparsky 26 Oct '21

26 Oct '21

While I love this idea, and I think this PEP is bound for glory, I don't love any of the proposed spellings for denoting late-binding arguments. It seems to me that a specialized symbol combination indicating that a particular argument has special behavior does not serve readability, regardless of what symbol combination is chosen. I would prefer to build on the fact that arguments already come in two flavors with somewhat different behaviors, and that the ordering of these is determined. Going by this analogy, it would make sense to have late-binding arguments following keyword arguments, set off by some separator, such as a double-pipe: def my_func(positional_arg, some_kwarg="foo" || late_binding_arg=[]]: # body of func or def my_func(positional_arg, some_kwarg="foo" || late_binding_arg=[]): # body of func I'm not massively attached to this particular symbol (which has advantages and disadvantages), but more to the idea that we can easily see that the arguments to the right of it are late-binding, and we can easily teach a beginner what that means. We can also easily scan a function definition and know that there is some specialized behavior going on, which is harder to see with a spelling like def my_func(positional_arg, some_kwarg<="foo", another_kwarg=[]): # body of func and we will probably even notice much more easily that there is a likely bug in that last definition (since we probably meant for the list arg to be late-binding) Best -Jon Kiparsky jon.kiparsky(a)gmail.com

3 2

Return False from __contains__ method if object not hashable for set and dict.
by Kazantcev Andrey 26 Oct '21

26 Oct '21

Now if do something like `[] in set()` python raise an exception, but if an object isn't hashable then we know for sure that it isn't in the set. Propose return False for these cases. What do you think?

3 2

def variable = value
by blek blek 25 Oct '21

25 Oct '21

Hello people, I was thinking of a "def" statement in Python. Its really weird that it can define only method, since it means "define". It shouldn't be hard to detect if this is a variable or a method, because method needs "()" to have arguments(even if there are zero of them), and defining variable value supposed to have "=" as the definition mark. Hope it would be added someday

5 6