Mailman 3 November 2021 - Python-ideas

zip(x, y, z, strict=True)
by Ram Rachum Dec. 1, 2022

Dec. 1, 2022

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

disallow assignment to unknown ssl.SSLContext attributes
by Thomas Grainger June 10, 2022

June 10, 2022

I was debugging some code that was using TLSv1.2 when I expected it to only support TLSv1.3, I tracked it down to a call to: context.miunimum_version = ssl.TLSVersion.TLSv1_3 it should have been: context.minimum_version = ssl.TLSVersion.TLSv1_3 I'd like invalid attribute assignment to be prevented at runtime

13 32

Bringing the print statement back
by Guido van Rossum March 2, 2022

March 2, 2022

In Python 3.10 we will no longer be burdened by the old parser (though 3rd party tooling needs to catch up). One thing that the PEG parser makes possible in about 20 lines of code is something not entirely different from the old print statement. I have a prototype: Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020, 16:31:17) [Clang 11.0.0 (clang-1100.0.33.8)] on darwin Type "help", "copyright", "credits" or "license" for more information. Cannot read termcap database; using dumb terminal settings. >>> print 2+2 4 >>> print "hello world" hello world >>> print "hello", input("Name:") Name:Guido hello Guido >>> print 1, 2, 3, sep=", " 1, 2, 3 >>> But wait, there's more! The same syntax will make it possible to call *any* function: >>> len "abc" 3 >>> Or any method: >>> import sys >>> sys.getrefcount "abc" 24 >>> Really, *any* method: >>> class C: ... def foo(self, arg): print arg ... >>> C().foo 2+2 4 >>> There are downsides too, though. For example, you can't call a method without arguments: >>> print <built-in function print> >>> Worse, the first argument cannot start with a parenthesis or bracket: >>> print (1, 2, 3) 1 2 3 >>> C().foo (1, 2, 3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: C.foo() takes 2 positional arguments but 4 were given >>> print (2+2), 42 4 (None, 42) >>> C().foo [0] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'method' object is not subscriptable >>> No, it's not April 1st. I am seriously proposing this (but I'll withdraw it if the response is a resounding "boo, hiss"). After all, we currently have a bunch of complexity in the parser just to give a helpful error message to people used to Python 2's print statement: >>> print 1, 2, 3 File "<stdin>", line 1 print 1, 2, 3 ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1, 2, 3)? >>> And IIRC there have been a number of aborted attempts at syntactic hacks to allow people to call functions (like print) without parentheses, although (I think) none of them made it into a PEP. The PEG parser makes this much simpler, because it can simply backtrack -- by placing the grammar rule for this syntax (tentatively called "call statement") last in the list of alternatives for "small statement" we ensure that everything that's a valid expression statement (including print() calls) is still an expression statement with exactly the same meaning, while still allowing parameter-less function calls, without lexical hacks. (There is no code in my prototype that checks for a space after 'print' -- it just checks that there's a name, number or string following a name, which is never legal syntax.) One possible extension I didn't pursue (yet -- dare me!) is to allow parameter-less calls inside other expressions. For example, my prototype does not support things like this: >>> a = (len "abc") File "<stdin>", line 1 a = (len "abc") ^ SyntaxError: invalid syntax >>> I think that strikes a reasonable balance between usability and reduced detection of common errors. I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo x' and we should only allow dotted names (sufficient to access functions in imported modules and method calls on variables). Or maybe we should only allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'. Or maybe we should really only bring back print statements. I believe there are some other languages that support a similar grammar (Ruby? R? Raku?) but I haven't investigated. Thoughts? -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him **(why is my pronoun here?)* <http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-…>

42 66

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 17, 2022

Feb. 17, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, such a class would look like this: class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify() push = heapq.heappush pop = heapq.heappop pushpop = heapq.heappushpop replace = heapq.heapreplace heapify = heapq.heapify # This could be a simple wrapper as well, but I had the following thoughts anyways, so here they are def nsmallest(self, n, key=None): # heapq.nsmallest makes a *max* heap of the first n elements, # while we know that self is already a min heap, so we can # make the max heap construction faster self[:n] = reversed(self[:n]) return heapq.nsmallest(n, self, key) # do we define nlargest on a minheap?? Wrapping around the C builtin functions (which aren't descriptors) would be a bit harder, but not much so: from functools import partial class Heap(list): def __init__(self, iterable=None): if iterable: super().__init__(iterable) else: super().__init__() self.heapify = partial(heapq.heapify, self) self.push = partial(heapq.heappush, self) ... self.heapify() Thoughts?

17 32

Allow syntax "func(arg=x if condition)"
by Peter O'Connor Jan. 19, 2022

Jan. 19, 2022

I often find that python lacks a nice way to say "only pass an argument under this condition". (See previous python-list email in "Idea: Deferred Default Arguments?") Example 1: Defining a list with conditional elements include_bd = True current_way = ['a'] + (['b'] if include_bd else [])+['c']+(['d'] if include_bd else []) new_way = ['a', 'b' if include_bd, 'c', 'd' if include_bd] also_new_way = list('a', 'b' if include_bd, 'c', 'd' if include_bd) Example 2: Deferring to defaults of called functions def is_close(a, b, precicion=1e-9): return abs(a-b) < precision def approach(pose, target, step=0.1, precision=None): # Defers to default precision if not otherwise specified: velocity = step*(target-pose) \ if not is_close(pose, target, precision if precision is not None) \ else 0 return velocity Not sure if this has been discussed, but I cannot see any clear downside to adding this, and it has some clear benefits (duplicated default arguments and **kwargs are the scourge of many real world code-bases)

6 9

Enhancing iterator objects with map, filter, reduce methods
by Remy Dec. 29, 2021

Dec. 29, 2021

Hi, I'd like to revive this thread after what seemed like a decade and see where this might take us 😃 I like this idea that the OP suggested but I'd like to extend this to all iterator objects (range_iterators, list_iterators etc.). 📌Idea Iterables to expose the .__iter__() method in iterable as .iter(). Iterators to implement 'transformational' functions like map, filter, flat_map, 'reductional' functions like reduce, sum, join, and 'evaluate' functions like to_list and to_set. ( [1,2,3].iter() .map(lambda x: x+1) .filter(lambda x: x % 2 == 0) .to_list() ) 📌Why? 1. It is readable in a manner that is procedural and dataflow centric. At one glance, it is easy to reason about how our data gets transformed - we start with the subject, our data, then we perform a sequence of transformations. The previous way of doing this: list( filter( lambda x: x%2==0, map( lambda x: x+1, iter([1,2,3])))) incurs a lot of cognitive overload. We could separate them into different lines: it = iter([1,2,3]) it = map(lambda x: x+1, it) it = filter(lambda x: x%2==0, it) list(it) but I would argue that there is a lot of repetition. 2. There are existing communities which deal with frameworks that focus on data transformations. This includes PySpark and pandas. 3. It is conventional - many modern languages like Rust, Scala, JavaScript and Kotlin have a similar API like this. 4. The existing map, filter APIs are not so commonly used (this is the impression I have). Apparently list comprehension appears more pythonic on StackOverflow posts. This extended iterator API could possibly 'revive' the use of lazy evaluation. 5. This API is 'flat', rather than the 'nested' map, reduce and filter. (I guess we can argue that "Flat is better than nested"?) 📌Why not? 1. Method chaining is uncommon and not really well-liked. I have this impression a lot among the community. Could somebody please explain why? 2. A lot of 'lambda' keywords. Agreed. (However, this would tie in well with the alternate lambda syntax proposal https://mail.python.org/archives/list/python-ideas@python.org/thread/QQUS35… .) 📌On list comprehension vs. method chaining I don't think the aim of this API should be to replace list comprehension and the like. Rather it offers programmers an alternative pattern to transform and reason about their data.

15 94

Re: Enhancing iterator objects with map, filter, reduce methods
by Matt D Nov. 30, 2021

Nov. 30, 2021

In your comprehension example, I'm fairly certain the filtering should be on the post incremented remainder [ x+1 for x in [1,2,3] if (x+1) % 2 == 0] I think that is the downside of the comprehension, having to repeat that twice. Maybe you can get around it with := ?? I'm stuck on 3.7 so I don't know for sure.

2 1

Magnitude and ProtoMagnitude ABCs — primarily for argument validation
by kana70416＠gmail.com Nov. 29, 2021

Nov. 29, 2021

Andrew Barnert via Python-ideas writes: ... Is there any commonly used or even imaginable useful type that uses them in weirder ways than set and float (which are both partially ordered) or np? array (where they aren’t even Boolean-values)? I've had occasion (a class for outcomes in two-player games) to define both < and <= as complete preorders, with "symmetry" in the sense that A < B iff B > A and A <= B iff B >= A, but < and <= were completely independent: they could be identical ("game of pure coordination"), they could be inverse ("game of pure competition"), and they could be anything else (eg, "prisoners' dilemma"). This system took a little getting used to, but writing "A > B and A >= B and A != B" [1] to implement "A Pareto dominates B" was one expressive convenience among others. I'm not sure this is "weird" in the sense you mean, and I greatly doubt this is sufficiently common to deserve an ABC :-), but it's a real example (long since archived on a disc whose exact location has slipped from memory ;-) that shows how flexible "ordering" in Python can be. Footnotes: [1] How the class, and specifically "A != B", were implemented is left for the reader who knows game theory to imagine. ;-)

1 0

easier writing to multiple streams
by Eyal Gruss Nov. 28, 2021

Nov. 28, 2021

hi first post here :) i have a recurring use pattern where i want to save to file everything i print to the screen. i have used the standard logging module and found it too cumbersome for my simple use case, and i find myself changing all my print()'s to custom myprint()'s. i am interested in suggesting several additions to the standard libraries, as i feel this use case is common enough, important enough, and currently painful enough to justify such changes, and that all four of the following suggestions could be useful in a complementary fashion. 1. allow the print() "file" argument to take a list of streams (currently it can take only a single stream). e.g: with open('output.txt', 'w') as f: print('hello', file=[sys.stdout, f]) this seems like the quickest and least resistant path for the user. need to decide how multiple streams will work with the flush argument. 2. modify contextlib.redirect_stdout() to take an additional boolean argument which we can call tee/replicate/duplicate/clone etc. when True it will duplicate instead of just redirecting: with open('output.txt', 'w') as f: with redirect_stdout(f, tee=True): # will duplicate instead of redirecting print('hello') or we could add a new context manager contextlib.copy_stdout() 3. allow the contextlib.redirect_stdout() "new_target" argument to take a list of streams, or allow specifying multiple arguments for multiple streams. e.g: with open('output.txt', 'w') as f: with redirect_stdout(sys.stdout, f): # or redirect_stdout([sys.stdout, f]) print('hello') this has the benefit over the tee argument of allowing more than one additional stream, but you need to explicitly specify stdout. so would be nice to have both modifications. 4. add to the standard io library a new class which gives you the write interface of a single stream, but is a wrapper that will write to multiple streams: with open('output.txt', 'w') as f: multi = io.OutputStreamCollection(sys.stdout, f) multi.write('hello\n') # or: print('hello', file=multi) this is similar to: https://stackoverflow.com/questions/9130755/wrapper-to-write-to-multiple-st…, and we need to decide about the rest of the stream methods. eyal.

8 8

PEP 671: Syntax for late-bound function argument defaults
by Chris Angelico Nov. 26, 2021

Nov. 26, 2021

Incorporates comments from the thread we just had. Is anyone interested in coauthoring this with me? Anyone who has strong interest in seeing this happen - whether you've been around the Python lists for years, or you're new and interested in getting involved for the first time, or anywhere in between! https://www.python.org/dev/peps/pep-0671/ PEP: 671 Title: Syntax for late-bound function argument defaults Author: Chris Angelico <rosuav(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 24-Oct-2021 Python-Version: 3.11 Post-History: 24-Oct-2021 Abstract ======== Function parameters can have default values which are calculated during function definition and saved. This proposal introduces a new form of argument default, defined by an expression to be evaluated at function call time. Motivation ========== Optional function arguments, if omitted, often have some sort of logical default value. When this value depends on other arguments, or needs to be reevaluated each function call, there is currently no clean way to state this in the function header. Currently-legal idioms for this include:: # Very common: Use None and replace it in the function def bisect_right(a, x, lo=0, hi=None, *, key=None): if hi is None: hi = len(a) # Also well known: Use a unique custom sentinel object _USE_GLOBAL_DEFAULT = object() def connect(timeout=_USE_GLOBAL_DEFAULT): if timeout is _USE_GLOBAL_DEFAULT: timeout = default_timeout # Unusual: Accept star-args and then validate def add_item(item, *optional_target): if not optional_target: target = [] else: target = optional_target[0] In each form, ``help(function)`` fails to show the true default value. Each one has additional problems, too; using ``None`` is only valid if None is not itself a plausible function parameter, the custom sentinel requires a global constant; and use of star-args implies that more than one argument could be given. Specification ============= Function default arguments can be defined using the new ``=>`` notation:: def bisect_right(a, x, lo=0, hi=>len(a), *, key=None): def connect(timeout=>default_timeout): def add_item(item, target=>[]): The expression is saved in its source code form for the purpose of inspection, and bytecode to evaluate it is prepended to the function's body. Notably, the expression is evaluated in the function's run-time scope, NOT the scope in which the function was defined (as are early-bound defaults). This allows the expression to refer to other arguments. Self-referential expressions will result in UnboundLocalError:: def spam(eggs=>eggs): # Nope Multiple late-bound arguments are evaluated from left to right, and can refer to previously-calculated values. Order is defined by the function, regardless of the order in which keyword arguments may be passed. Choice of spelling ------------------ Our chief syntax proposal is ``name=>expression`` -- our two syntax proposals ... ahem. Amongst our potential syntaxes are:: def bisect(a, hi=>len(a)): def bisect(a, hi=:len(a)): def bisect(a, hi?=len(a)): def bisect(a, hi!=len(a)): def bisect(a, hi=\len(a)): def bisect(a, hi=`len(a)`): def bisect(a, hi=@len(a)): Since default arguments behave largely the same whether they're early or late bound, the preferred syntax is very similar to the existing early-bind syntax. The alternatives offer little advantage over the preferred one. How to Teach This ================= Early-bound default arguments should always be taught first, as they are the simpler and more efficient way to evaluate arguments. Building on them, late bound arguments are broadly equivalent to code at the top of the function:: def add_item(item, target=>[]): # Equivalent pseudocode: def add_item(item, target=<OPTIONAL>): if target was omitted: target = [] Open Issues =========== - yield/await? Will they cause problems? Might end up being a non-issue. - annotations? They go before the default, so is there any way an anno could want to end with ``=>``? References ========== Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

31 292