Mailman 3 January 2022 - Python-ideas

PEP 671 (late-bound arg defaults), next round of discussion!
by Chris Angelico 25 Dec '22

25 Dec '22

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ with some additional information about the reference implementation, and some clarifications elsewhere. *PEP 671: Syntax for late-bound function argument defaults* Questions, for you all: 1) If this feature existed in Python 3.11 exactly as described, would you use it? 2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden? (It's absolutely valid to say "yes" and "yes", and feel free to say which of those pulls is the stronger one.) 3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else? 4) If "no" to question 1, is there some other spelling or other small change that WOULD mean you would use it? (Some examples in the PEP.) 5) Do you know how to compile CPython from source, and would you be willing to try this out? Please? :) I'd love to hear, also, from anyone's friends/family who know a bit of Python but haven't been involved in this discussion. If late-bound defaults "just make sense" to people, that would be highly informative. Any and all comments welcomed. I mean, this is python-ideas after all... bikeshedding is what we do best! The reference implementation currently has some test failures, which I'm looking into. I'm probably going to make this my personal default Python interpreter for a while, to see how things go. ChrisA

40 353

zip(x, y, z, strict=True)
by Ram Rachum 01 Dec '22

01 Dec '22

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

disallow assignment to unknown ssl.SSLContext attributes
by Thomas Grainger 10 Jun '22

10 Jun '22

I was debugging some code that was using TLSv1.2 when I expected it to only support TLSv1.3, I tracked it down to a call to: context.miunimum_version = ssl.TLSVersion.TLSv1_3 it should have been: context.minimum_version = ssl.TLSVersion.TLSv1_3 I'd like invalid attribute assignment to be prevented at runtime

13 32

Bringing the print statement back
by Guido van Rossum 02 Mar '22

02 Mar '22

In Python 3.10 we will no longer be burdened by the old parser (though 3rd party tooling needs to catch up). One thing that the PEG parser makes possible in about 20 lines of code is something not entirely different from the old print statement. I have a prototype: Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020, 16:31:17) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. Cannot read termcap database; using dumb terminal settings. >>> print 2+2 4 >>> print "hello world" hello world >>> print "hello", input("Name:") Name:Guido hello Guido >>> print 1, 2, 3, sep=", " 1, 2, 3 >>> But wait, there's more! The same syntax will make it possible to call *any* function: >>> len "abc" 3 >>> Or any method: >>> import sys >>> sys.getrefcount "abc" 24 >>> Really, *any* method: >>> class C: ... def foo(self, arg): print arg ... >>> C().foo 2+2 4 >>> There are downsides too, though. For example, you can't call a method without arguments: >>> print <built-in function print> >>> Worse, the first argument cannot start with a parenthesis or bracket: >>> print (1, 2, 3) 1 2 3 >>> C().foo (1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: C.foo() takes 2 positional arguments but 4 were given >>> print (2+2), 42 4 (None, 42) >>> C().foo [0] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'method' object is not subscriptable >>> No, it's not April 1st. I am seriously proposing this (but I'll withdraw it if the response is a resounding "boo, hiss"). After all, we currently have a bunch of complexity in the parser just to give a helpful error message to people used to Python 2's print statement: >>> print 1, 2, 3 File "<stdin>", line 1 print 1, 2, 3 ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1, 2, 3)? >>> And IIRC there have been a number of aborted attempts at syntactic hacks to allow people to call functions (like print) without parentheses, although (I think) none of them made it into a PEP. The PEG parser makes this much simpler, because it can simply backtrack -- by placing the grammar rule for this syntax (tentatively called "call statement") last in the list of alternatives for "small statement" we ensure that everything that's a valid expression statement (including print() calls) is still an expression statement with exactly the same meaning, while still allowing parameter-less function calls, without lexical hacks. (There is no code in my prototype that checks for a space after 'print' -- it just checks that there's a name, number or string following a name, which is never legal syntax.) One possible extension I didn't pursue (yet -- dare me!) is to allow parameter-less calls inside other expressions. For example, my prototype does not support things like this: >>> a = (len "abc") File "<stdin>", line 1 a = (len "abc") ^ SyntaxError: invalid syntax >>> I think that strikes a reasonable balance between usability and reduced detection of common errors. I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo x' and we should only allow dotted names (sufficient to access functions in imported modules and method calls on variables). Or maybe we should only allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'. Or maybe we should really only bring back print statements. I believe there are some other languages that support a similar grammar (Ruby? R? Raku?) but I haven't investigated. Thoughts? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

42 66

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow 17 Feb '22

17 Feb '22

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, such a class would look like this: class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify() push = heapq.heappush pop = heapq.heappop pushpop = heapq.heappushpop replace = heapq.heapreplace heapify = heapq.heapify # This could be a simple wrapper as well, but I had the following thoughts anyways, so here they are def nsmallest(self, n, key=None): # heapq.nsmallest makes a *max* heap of the first n elements, # while we know that self is already a min heap, so we can # make the max heap construction faster self[:n] = reversed(self[:n]) return heapq.nsmallest(n, self, key) # do we define nlargest on a minheap?? Wrapping around the C builtin functions (which aren't descriptors) would be a bit harder, but not much so: from functools import partial class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify = partial(heapq.heapify, self) self.push = partial(heapq.heappush, self) ... self.heapify() Thoughts?

17 32

Revisiting a frozenset display literal
by Steven D'Aprano 31 Jan '22

31 Jan '22

Inspired by this enhancement request: https://bugs.python.org/issue46393 I thought it might be time to revist the idea of a frozenset display. This has been discussed a few times before, such as here: https://mail.python.org/pipermail/python-ideas/2018-July/051902.html We have displays for the most important builtin data structures: - lists [1, 2, 3] - tuples (1, 2, 3) - dicts {1: 0, 2: 0, 3: 0} - sets {1, 2, 3} (as well as literals for ints, floats, strings and bytes) but not for frozensets. So the only way to guarantee that you have a frozenset is to explicitly call the builtin, which is not only a runtime call rather than a compile-time operation, but can be monkey-patched or shadowed. CPython already has some neat optimizations in place to use frozensets instead of sets, for example: >>> import dis >>> dis.dis("x in {1, 2, 3}") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 0 (frozenset({1, 2, 3})) 4 CONTAINS_OP 0 6 RETURN_VALUE and the compiler can build frozensets of literals as a constant. Ironically, this means that actually making a frozenset explicitly does far more work than needed: >>> dis.dis("frozenset({1, 2, 3})") 1 0 LOAD_NAME 0 (frozenset) 2 BUILD_SET 0 4 LOAD_CONST 0 (frozenset({1, 2, 3})) 6 SET_UPDATE 1 8 CALL_FUNCTION 1 10 RETURN_VALUE Got that? To create a frozenset of literals, first the compiler creates a frozenset constant containing what you wanted. Then at runtime, it: - looks up frozenset in globals and builtins; - loads the pre-prepared frozenset (which is exactly what we want); - creates a new set from that frozenset; - calls the frozenset() function on that set to create a new frozenset that duplicates the pre-prepared one; - and finally garbage-collects the temporary set. So to get the frozenset we want, we start with the frozenset we want, and make an unnecessary copy the long way o_O If you didn't know that every step in that song and dance routine was necessary, it would seem ludicrously wasteful. If we had a frozenset display, we could avoid most of that work, optimizing that down to a LOAD_CONST like this: >>> dis.dis('(1, 2, 3)') 1 0 LOAD_CONST 0 ((1, 2, 3)) 2 RETURN_VALUE It seems to me that all of the machinery to make this work already exists. The compiler already knows how to create frozensets at compile-time, avoiding the need to lookup and call the frozenset() builtin. All we need is syntax for a frozenset display. How does this work for you? f{1, 2, 3} -- Steve

26 114

Limit scope of variables using round brackets
by Iyad Ahmed 29 Jan '22

29 Jan '22

Example use cases: - Loop variables and similar, it is sometimes more comfortable to scope it to the loop body only - Not having to name variables var1, var2, var3, or makeup unnecessarily unique variable names, in some situations In other languages the curly bracket is used, in Python round brackets might fit more in the syntax. I don’t have much experience with the implementation of Python, so not sure if there’s a technical reason against such idea, feedback is very appreciated Best regards, Iyad Ahmed Email: iyadahmed(a)cgonfire.com Socials: Twitter <https://twitter.com/cgonfire> GitHub: https://github.com/iyadahmed

8 11

'frozen set' discussion - please consider taking some time off.
by C. Titus Brown 21 Jan '22

21 Jan '22

Hi all, python-ideas moderator here. It’d be great if y’all could take a few days to cool off the frozen set discussion, which is veering off the rails a little bit into emotional language. I’ll keep an eye on it and put emergency moderation into effect if I must, but it’d be nicer if I didn’t feel the need to do that. thanks, —titus

1 0

Allowing non-ASCII bracket and quote characters in source code
by John Sturdy 21 Jan '22

21 Jan '22

Perhaps the time isn't ripe for this, and perhaps it never will be, but UTF8 seems to be handled by just about everything these days. I suspect this is a crazy suggestion, but on the other hand perhaps people looking back from 2100 will think "It was crazy that they stuck exclusively with ASCII syntax characters for so long after Unicode was widely available". Sometimes when you have quite a few levels of brackets, and there are more than one of the same type, might it be better to allow variants of each type of bracket character? Unicode provides bold, double, small, superscript, subscript, and white-filled (hollow) variants of round, square and curly brackets, and top and bottom ticked variants of square brackets. Perhaps not enough platforms will be able to display them? And entering them may be fiddly, although programmable editors and IDEs could let you type the standard characters but pick variants on a round-robin basis within each expression. But it might be handled better in the display in editors and IDEs (perhaps syntax coloring brackets by their depth). Or some might say not to write such deeply nested bracketed expressions. Implementation parsers could simply translate all the brackets to the base types, or they could treat them as equivalent but distinct, and check that the open and close brackets match, which might catch a few errors. There are also other quote characters available, such as the guillemets traditionally used in French. There's absolutely no need to use such things, but for people working on code which will be used internally in, say, French or Quebecois organizations might welcome it. OK, shoot this one down now :-)

13 19

Re: Revisiting a frozenset display literal
by Steven D'Aprano 19 Jan '22

19 Jan '22

On Wed, Jan 19, 2022 at 01:56:54PM +0100, Marco Sulla wrote: > PEP 351 for the frozen protocol was rejected. I didn't read why, but > it's probably hard to resurrect. This is not a proposal for a generic frozen protocol. It is a simple proposal for a set method that returns a frozenset, with the aim that the peephole optimiser will be able to optimise the call out for set displays consisting only of literals. -- Steve

3 3

Python-ideas January 2022