Mailman 3 March 2022 - Python-ideas

PEP 671 (late-bound arg defaults), next round of discussion!
by Chris Angelico 23 Jul '23

23 Jul '23

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ with some additional information about the reference implementation, and some clarifications elsewhere. *PEP 671: Syntax for late-bound function argument defaults* Questions, for you all: 1) If this feature existed in Python 3.11 exactly as described, would you use it? 2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden? (It's absolutely valid to say "yes" and "yes", and feel free to say which of those pulls is the stronger one.) 3) If "yes" to question 1, would you use it for any/all of (a) mutable defaults, (b) referencing things that might have changed, (c) referencing other arguments, (d) something else? 4) If "no" to question 1, is there some other spelling or other small change that WOULD mean you would use it? (Some examples in the PEP.) 5) Do you know how to compile CPython from source, and would you be willing to try this out? Please? :) I'd love to hear, also, from anyone's friends/family who know a bit of Python but haven't been involved in this discussion. If late-bound defaults "just make sense" to people, that would be highly informative. Any and all comments welcomed. I mean, this is python-ideas after all... bikeshedding is what we do best! The reference implementation currently has some test failures, which I'm looking into. I'm probably going to make this my personal default Python interpreter for a while, to see how things go. ChrisA

43 364

zip(x, y, z, strict=True)
by Ram Rachum 01 Dec '22

01 Dec '22

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

disallow assignment to unknown ssl.SSLContext attributes
by Thomas Grainger 10 Jun '22

10 Jun '22

I was debugging some code that was using TLSv1.2 when I expected it to only support TLSv1.3, I tracked it down to a call to: context.miunimum_version = ssl.TLSVersion.TLSv1_3 it should have been: context.minimum_version = ssl.TLSVersion.TLSv1_3 I'd like invalid attribute assignment to be prevented at runtime

13 32

mro and super don't feel so pythonic
by malmiteria 26 Apr '22

26 Apr '22

Hi, Before anything, i made a github repository about this topic here : https://github.com/malmiteria/super-alternative-to-super The core of what i wanna discuss here is that i don't think mro and super (mainly because it relies on mro) are very pythonic. Mainly that some behaviors of the mro are too implicit, and are silencing what really should be errors. Let me explain : in case of multiple inheritence, resolving a child method from it's parent isn't an obvious task, and mro comes as a solution to that. However, i don't understand why we don't let the programmer solve it. I think this is similar to a merge conflict, and not letting the programmer resolve the conflict feels like silencing an error. This is especially infuriating when you realise that mro doesn't solve all possible scenarios, and then, simply refuses the opportunity to solve it to the programmer. Then, super relying on mro gives off some weird behaviors, mainly, it's possible for a child definition to affect what a call to super means in it's parent. This feels like a side effect (which is the 'too implicit' thing i refer to). I also don't understand why we can't simply pass the parent targeted as argument to super, instead of having no argument, or having to pass the current class and instances as argument : super(child) is a proxy to parent, when super(parent) would make more sense to be the proxy to parent, in my mind. I dive in more depths about those topics in the readme of the github repository i linked at the top of this comment. what i propose is a solution that would follow those rules: The mro alternative, which i called explicit method resolution aka EMR (which is probably not a good name since i apply it, as mro, to all class attributes), follow those rules : 1) Straightforward case : the class definition has the method / attribute : this is the one EMR should resolve to 2) Not found : the method / attribute can't be resolved in the class itself, or by any of it's parents, then it should raise an AttributeError 3) Only on parent : the method / attribute can't be resolved in the class itself, and can only be resolved by one of it's parents, this is the one EMR should resolve to 4) Multiple parent : the method / attribute can't be resolved in the class itself, and can be resolved by at least 2 of it's parents, then an ExplicitResolutionRequired error should be raised 5) Transimittin errors : the method / attribute can't be resolved in the class itself, and one parent at least raises an ExplicitResolutionRequired error, then it should raise an ExplicitResolutionRequired error 6) (optional?) Single source : when multiple parent can resolve a method from a single source (in case of diamond shape inheritence), the ExplicitResolutionRequired is not needed The super alternative, which i called __as_parent__ should follow those rules : 1) reliability : the target __as_parent__ points to should not depend on anything other than the argument passed to it 2) expliciteness : in case of multiple inheritence, the parent targetted should be passed as an argument to the __as_parent__ method. 3) impliciteness : in case of simple inheritence, it is not needed to specify the parent targeted (since there can only be one, and it make it closer to the actual behavior of super in most cases) 4) ancestors as targets : should be able to target ancestors, either direct or not (which is needed in case two grandparent define a method that a single parent share, there would be no other way to solve the ExplicitResolutionRequired otherwise) this solution has a few advantages in my mind : - the current mro and super are more tightly coupled than the emr and __as_parent__ i propose here - the __as_parent__ is more reliable than super in its behavior, and should lead to an easier learning curve - the emr i propose as a replacement to mro allows for some inheritence tree mro doesn't allow. - the __as_parent__ method being able to target specific parent allows for different methods to visit the parents in different order easily, which today would be harder, since the parent visiting order is tightly coupled to the class definition - with emr, in case of problematic resolution, an error is raised to tell you about the problem, and ask you for explicit resolution, the current solution doesn't, which can lead to surprises closer to production environment. A few possible downsides : - the transition would be a pain to deal with - the current mro allows for dependencies injection in the inheritence tree. I believe this feature should be untied from super and mro, but it would require some work. - the current super and mro are old enough to have been faced with issues and having been updated to solve those issues down the line. Any alternative would have to face those issues again and new ones down the line - any coexistence between those two solution would require some work to make sure they don't break one another (i've explored some of those scenarios in my repository, but i definitely didn't cover it all) I believe that what i talk about here is definitely too much at once. For exemple, adding a kwarg to super to specify the parent it targets would be a very easy change to add into python, and wouldn't require much more of what i talk about here, but it would still have some of the value i talk about here. But overall, it all makes sense to me, and i wanted to share it all with you guys. What do you think? does it makes sense? Am i missing something?

22 206

An unambiguous way of initializing an empty set and dictionary
by joao.p.f.batista.97＠gmail.com 06 Apr '22

06 Apr '22

Currently: l = [] # new empty list t = () # new empty tuple s = set() # new empty set (no clean and consistent way of initializing regarding the others) <<< d = {} # new empty dictionary Possible solution: s = {} # new empty set d = {:} # new empty dictionary (the ":" is a reference to key-value pairs) Current workaround at least for consistency: l = list() # new empty list t = tuple() # new empty tuple s = set() # new empty set d = dict() # new empty dictionary However, it doesn't feel right to not be able to initialize an empty set as cleanly and consistently as lists, tuples and dictionaries in both forms.

8 12

Allow os.getenv to optionally convert result to boolean
by Adrian Torres Justo 06 Apr '22

06 Apr '22

We've used a lot of boolean environment variables in a recent project I've been working on, this led us to create a wrapper to os.getenv that essentially converts the result to a boolean using distutils.util.strtobool if an optional kwarg to our wrapper is set to True. Would it be far-fetched to have os.getenv incorporate this functionality? We already have the building blocks (strtobool and getenv), and the signature change would be minimal and backwards-compatible: def getenv(key, default=None, to_bool=False): ... The implementation would be trivial too. Let me know what you think, thanks! Adrian

5 9

Less is more? Smaller code and data to fit more into the CPU cache?
by Jonathan Fine 05 Apr '22

05 Apr '22

Hi As you may have seen, AMD has recently announced CPUs that have much larger L3 caches. Does anyone know of any work that's been done to research or make critical Python code and data smaller so that more of it fits in the CPU cache? I'm particularly interested in measured benefits. This search https://www.google.com/search?q=python+performance+CPU+cache+size provides two relevant links https://www.oreilly.com/library/view/high-performance-python/9781449361747/… https://www.dlr.de/sc/Portaldata/15/Resources/dokumente/pyhpc2016/slides/Py… but not much else I found relevant. AnandTech writes about the chips with triple the L3 cache: https://www.anandtech.com/show/17323/amd-releases-milan-x-cpus-with-3d-vcac… "As with other chips that incorporate larger caches, the greatest benefits are going to be found in workloads that spill out of contemporary-sized caches, but will neatly fit into the larger cache." And also: https://www.anandtech.com/show/17313/ryzen-7-5800x3d-launches-april-20th-pl… " As detailed by the company back at CES 2022 and reiterated in today’s announcement, AMD has found that the chip is 15% faster at gaming than their Ryzen 9 5900X." I already know that using Non Uniform Memory Access (NUMA) raises the difficult problem of cache coherence. https://en.wikipedia.org/wiki/Non-uniform_memory_access https://en.wikipedia.org/wiki/Cache_coherence -- Jonathan

5 6

Syntax proposal of for..in..if in regular for loops
by Svein Seldal 04 Apr '22

04 Apr '22

I'm sorry for reposting, but this message got stuck in moderation approval for 5 days so I figured I should try again. I'd like to propose extending the for statement to include conditionals akin to comprehensions in order to simplify for loop statements: `for .. in .. if ..:` E.g. for x in y if x in c: some_op(x) which is functionally equivalent as for x in y: if x not in c: continue some_op(x) The `for .. in .. if` syntax is a well-known construct from list comprehension syntax [1]. Other alternative ways to do this with list comprehension is: for x in (a for a in y if c): or it = (a for a in y if c) for x in it: Without having examined all use cases, I believe the same syntax should be applied this syntax as for asynchronous comprehensions. [1] PEP 202 - List Comprehensions [2] PEP 530 - Asynchronous Comprehensions Best regards, Svein Seldal

20 65

A string function idea
by StrikerOmega 30 Mar '22

30 Mar '22

Hello everyone, When I am coding in Python I often encounter situations where I have a string like this one. sample=""" fruit:apple tree:[Apple tree] quantity:{5} quantity:{3} """ And I want to grab some kind of value from it. So let me introduce you to the grab function. This is the definition: def grab(start, end, list_out=False): Now, if you want to get the fruit from the sample string you can just "grab" it as follows: sample.grab(start="fruit:", end="\n") >> 'apple' You can also "grab" values enclosed in brackets or in any kind of character and It works as you would expect. sample.grab(start="tree:[", end="]") >> 'Apple tree' The optional argument "list_out" makes the function return a list of values, for cases where there are several of them to grab. sample.grab(start="quantity:{", end="}", list_out=True) >> [5, 3] The "grab" function will return only the first occurrence if "list_out" is omitted or passed as False. sample.grab(start="quantity:{", end="}") >> 5 sample.grab(start="quantity:{", end="}", list_out=False) >> 5 As you can see, it is incredibly useful for extracting substrings from a larger and more complex string and can change the way we parse strings for the better. For example, it could simplify the way we parse the fields of HTTP headers. It also has many applications for the parsing of configuration files and the construction of strings for SQL queries and bash commands among others. I have implemented this function locally to use it for my personal projects and It has proven to be really useful in practice, so I hope you find it useful as well and consider adding it as a class method for strings in Python. Julio Cabria Engineering student Autonomous University of Madrid

11 20

Anonymous namedtuples, revisited
by michael＠michaelgreen.dev 29 Mar '22

29 Mar '22

Back in early 2016 there was an idea submitted[1] by Joseph Martinot-Lagarde in regard to anonymous namedtuples. The crux of the submission was the idea of instantiating a namedtuple using only Python's generic tuple syntax plus keyword arguments. At the time, the major rebuttal against this idea however was that, since kwargs are an unordered collection, it would be non-trivial to figure out a proper convention for mapping an unordered collection onto an ordered one. Times have changed however, and with python3.6/3.7 we now have the guarantee that dictionaries are insertion ordered. This means that by extension keyword arguments are also ordered, and thus the major point of contest against anonymous namedtuples is now moot. And as such I would like to once again suggest anonymous namedtuples as a potential PEP. Proposal ======= The proposal ultimately is the same as before: ``` my_point = (x=12, y=16) # (x=12, y=16) my_point[0] # 12 ``` where kwargs can be passed to a tuple's instantiation [either via tuple() or just ()], and these tuples would have all the same properties of a namedtuple. A basic, pure-python implementation is actually quite trivial now given that kwargs are ordered. Observe: ``` def atuple(**kwargs): signature = " ".join(kwargs) _type = atuple.__dict__.get(signature) if not _type: _type = namedtuple(atuple.__name__, signature) atuple.__dict__[signature] = _type return _type(**kwargs) ``` This has none of the problems suggested in the previous rebuttal. anonymous tuples of incorrect ordering raise the assertion error, as expected: ``` >>> assert atuple(x=12, y=16) == atuple(y=16, x=12) Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError ``` ...and by extension it preserves the users's specified ordering as given by **kwargs. ``` >>> pt = atuple(r=3, theta=0.5, phi=0.25) >>> pt atuple(r=3, theta=0.5, phi=0.25) ``` Given that this implementation caches the type by the keyword signature, anonymous tuples of the same signature are recognized as the same type. However, given differing values, they are not equal. ``` >>> type(atuple(x=1,y=2)) == type(atuple(x=1,y=2)) True >>> atuple(x=1,y=2) == atuple(x=1,y=3) False ``` This allows us to provide positional arguments to the construct, just like regular namedtuples, and passing a mismatch throws an error as expected. ``` >>> pt = atuple(x=12, y=16) >>> pt atuple(x=12, y=16) >>> type(pt)(1,2) atuple(x=1, y=2) >>> type(pt)(1,2,3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: atuple.__new__() takes 3 positional arguments but 4 were given ``` As we can see, the introduction of insertion-ordered dictionaries solves all of the issues previously raised with regard to anonymous namedtuples. References: [1] https://mail.python.org/pipermail/python-ideas/2016-April/039857.html

10 17