Mailman 3 March 2022 - Python-ideas

PEP 671 (late-bound arg defaults), next round of discussion!
by Chris Angelico July 23, 2023

July 23, 2023

I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/ with some additional information about the reference implementation, and some clarifications elsewhere. *PEP 671: Syntax for late-bound function argument defaults* Questions, for you all: 1) If this feature existed in Python 3.11 exactly as described, would you use it? 2) Independently: Is the syntactic distinction between "=" and "=>" a cognitive burden? (It's absolutely valid to say "yes" and "yes", and feel free to … [View More]

43 364

zip(x, y, z, strict=True)
by Ram Rachum Dec. 1, 2022

Dec. 1, 2022

Here's something that would have saved me some debugging yesterday: >>> zipped = zip(x, y, z, strict=True) I suggest that `strict=True` would ensure that all the iterables have been exhausted, raising an exception otherwise. This is useful in cases where you're assuming that the iterables all have the same lengths. When your assumption is wrong, you currently just get a shorter result, and it could take you a while to figure out why it's happening. What do you think?

27 179

disallow assignment to unknown ssl.SSLContext attributes
by Thomas Grainger June 10, 2022

June 10, 2022

I was debugging some code that was using TLSv1.2 when I expected it to only support TLSv1.3, I tracked it down to a call to: context.miunimum_version = ssl.TLSVersion.TLSv1_3 it should have been: context.minimum_version = ssl.TLSVersion.TLSv1_3 I'd like invalid attribute assignment to be prevented at runtime

13 32

mro and super don't feel so pythonic
by malmiteria April 26, 2022

April 26, 2022

Hi, Before anything, i made a github repository about this topic here : https://github.com/malmiteria/super-alternative-to-super The core of what i wanna discuss here is that i don't think mro and super (mainly because it relies on mro) are very pythonic. Mainly that some behaviors of the mro are too implicit, and are silencing what really should be errors. Let me explain : in case of multiple inheritence, resolving a child method from it's parent isn't an obvious task, and mro comes as a … [View More]solution to that. However, i don't understand why we don't let the programmer solve it. I think this is similar to a merge conflict, and not letting the programmer resolve the conflict feels like silencing an error. This is especially infuriating when you realise that mro doesn't solve all possible scenarios, and then, simply refuses the opportunity to solve it to the programmer. Then, super relying on mro gives off some weird behaviors, mainly, it's possible for a child definition to affect what a call to super means in it's parent. This feels like a side effect (which is the 'too implicit' thing i refer to). I also don't understand why we can't simply pass the parent targeted as argument to super, instead of having no argument, or having to pass the current class and instances as argument : super(child) is a proxy to parent, when super(parent) would make more sense to be the proxy to parent, in my mind. I dive in more depths about those topics in the readme of the github repository i linked at the top of this comment. what i propose is a solution that would follow those rules: The mro alternative, which i called explicit method resolution aka EMR (which is probably not a good name since i apply it, as mro, to all class attributes), follow those rules : 1) Straightforward case : the class definition has the method / attribute : this is the one EMR should resolve to 2) Not found : the method / attribute can't be resolved in the class itself, or by any of it's parents, then it should raise an AttributeError 3) Only on parent : the method / attribute can't be resolved in the class itself, and can only be resolved by one of it's parents, this is the one EMR should resolve to 4) Multiple parent : the method / attribute can't be resolved in the class itself, and can be resolved by at least 2 of it's parents, then an ExplicitResolutionRequired error should be raised 5) Transimittin errors : the method / attribute can't be resolved in the class itself, and one parent at least raises an ExplicitResolutionRequired error, then it should raise an ExplicitResolutionRequired error 6) (optional?) Single source : when multiple parent can resolve a method from a single source (in case of diamond shape inheritence), the ExplicitResolutionRequired is not needed The super alternative, which i called __as_parent__ should follow those rules : 1) reliability : the target __as_parent__ points to should not depend on anything other than the argument passed to it 2) expliciteness : in case of multiple inheritence, the parent targetted should be passed as an argument to the __as_parent__ method. 3) impliciteness : in case of simple inheritence, it is not needed to specify the parent targeted (since there can only be one, and it make it closer to the actual behavior of super in most cases) 4) ancestors as targets : should be able to target ancestors, either direct or not (which is needed in case two grandparent define a method that a single parent share, there would be no other way to solve the ExplicitResolutionRequired otherwise) this solution has a few advantages in my mind : - the current mro and super are more tightly coupled than the emr and __as_parent__ i propose here - the __as_parent__ is more reliable than super in its behavior, and should lead to an easier learning curve - the emr i propose as a replacement to mro allows for some inheritence tree mro doesn't allow. - the __as_parent__ method being able to target specific parent allows for different methods to visit the parents in different order easily, which today would be harder, since the parent visiting order is tightly coupled to the class definition - with emr, in case of problematic resolution, an error is raised to tell you about the problem, and ask you for explicit resolution, the current solution doesn't, which can lead to surprises closer to production environment. A few possible downsides : - the transition would be a pain to deal with - the current mro allows for dependencies injection in the inheritence tree. I believe this feature should be untied from super and mro, but it would require some work. - the current super and mro are old enough to have been faced with issues and having been updated to solve those issues down the line. Any alternative would have to face those issues again and new ones down the line - any coexistence between those two solution would require some work to make sure they don't break one another (i've explored some of those scenarios in my repository, but i definitely didn't cover it all) I believe that what i talk about here is definitely too much at once. For exemple, adding a kwarg to super to specify the parent it targets would be a very easy change to add into python, and wouldn't require much more of what i talk about here, but it would still have some of the value i talk about here. But overall, it all makes sense to me, and i wanted to share it all with you guys. What do you think? does it makes sense? Am i missing something? [View Less]

22 206

An unambiguous way of initializing an empty set and dictionary
by joao.p.f.batista.97＠gmail.com April 6, 2022

April 6, 2022

Currently: l = [] # new empty list t = () # new empty tuple s = set() # new empty set (no clean and consistent way of initializing regarding the others) <<< d = {} # new empty dictionary Possible solution: s = {} # new empty set d = {:} # new empty dictionary (the ":" is a reference to key-value pairs) Current workaround at least for consistency: l = list() # new empty list t = tuple() # new empty tuple s = set() # new empty set d = dict() # new empty dictionary However, it doesn't … [View More]

8 12

Allow os.getenv to optionally convert result to boolean
by Adrian Torres Justo April 6, 2022

April 6, 2022

We've used a lot of boolean environment variables in a recent project I've been working on, this led us to create a wrapper to os.getenv that essentially converts the result to a boolean using distutils.util.strtobool if an optional kwarg to our wrapper is set to True. Would it be far-fetched to have os.getenv incorporate this functionality? We already have the building blocks (strtobool and getenv), and the signature change would be minimal and backwards-compatible: def getenv(key, default=… [View More]

5 9

Less is more? Smaller code and data to fit more into the CPU cache?
by Jonathan Fine April 5, 2022

April 5, 2022

Hi As you may have seen, AMD has recently announced CPUs that have much larger L3 caches. Does anyone know of any work that's been done to research or make critical Python code and data smaller so that more of it fits in the CPU cache? I'm particularly interested in measured benefits. This search https://www.google.com/search?q=python+performance+CPU+cache+size provides two relevant links https://www.oreilly.com/library/view/high-performance-python/9781449361747/… https://www.dlr.de/sc/… [View More]

5 6

Syntax proposal of for..in..if in regular for loops
by Svein Seldal April 4, 2022

April 4, 2022

I'm sorry for reposting, but this message got stuck in moderation approval for 5 days so I figured I should try again. I'd like to propose extending the for statement to include conditionals akin to comprehensions in order to simplify for loop statements: `for .. in .. if ..:` E.g. for x in y if x in c: some_op(x) which is functionally equivalent as for x in y: if x not in c: continue some_op(x) The `for .. in .. if` syntax is a … [View More]

20 65

A string function idea
by StrikerOmega March 30, 2022

March 30, 2022

Hello everyone, When I am coding in Python I often encounter situations where I have a string like this one. sample=""" fruit:apple tree:[Apple tree] quantity:{5} quantity:{3} """ And I want to grab some kind of value from it. So let me introduce you to the grab function. This is the definition: def grab(start, end, list_out=False): Now, if you want to get the fruit from the sample string you can just "grab" it as follows: sample.grab(start="fruit:", end="\n") >> 'apple' You can … [View More]

11 20

Anonymous namedtuples, revisited
by michael＠michaelgreen.dev March 29, 2022

March 29, 2022

Back in early 2016 there was an idea submitted[1] by Joseph Martinot-Lagarde in regard to anonymous namedtuples. The crux of the submission was the idea of instantiating a namedtuple using only Python's generic tuple syntax plus keyword arguments. At the time, the major rebuttal against this idea however was that, since kwargs are an unordered collection, it would be non-trivial to figure out a proper convention for mapping an unordered collection onto an ordered one. Times have changed … [View More]however, and with python3.6/3.7 we now have the guarantee that dictionaries are insertion ordered. This means that by extension keyword arguments are also ordered, and thus the major point of contest against anonymous namedtuples is now moot. And as such I would like to once again suggest anonymous namedtuples as a potential PEP. Proposal ======= The proposal ultimately is the same as before: ``` my_point = (x=12, y=16) # (x=12, y=16) my_point[0] # 12 ``` where kwargs can be passed to a tuple's instantiation [either via tuple() or just ()], and these tuples would have all the same properties of a namedtuple. A basic, pure-python implementation is actually quite trivial now given that kwargs are ordered. Observe: ``` def atuple(**kwargs): signature = " ".join(kwargs) _type = atuple.__dict__.get(signature) if not _type: _type = namedtuple(atuple.__name__, signature) atuple.__dict__[signature] = _type return _type(**kwargs) ``` This has none of the problems suggested in the previous rebuttal. anonymous tuples of incorrect ordering raise the assertion error, as expected: ``` >>> assert atuple(x=12, y=16) == atuple(y=16, x=12) Traceback (most recent call last): File "<stdin>", line 1, in <module> AssertionError ``` ...and by extension it preserves the users's specified ordering as given by **kwargs. ``` >>> pt = atuple(r=3, theta=0.5, phi=0.25) >>> pt atuple(r=3, theta=0.5, phi=0.25) ``` Given that this implementation caches the type by the keyword signature, anonymous tuples of the same signature are recognized as the same type. However, given differing values, they are not equal. ``` >>> type(atuple(x=1,y=2)) == type(atuple(x=1,y=2)) True >>> atuple(x=1,y=2) == atuple(x=1,y=3) False ``` This allows us to provide positional arguments to the construct, just like regular namedtuples, and passing a mismatch throws an error as expected. ``` >>> pt = atuple(x=12, y=16) >>> pt atuple(x=12, y=16) >>> type(pt)(1,2) atuple(x=1, y=2) >>> type(pt)(1,2,3) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: atuple.__new__() takes 3 positional arguments but 4 were given ``` As we can see, the introduction of insertion-ordered dictionaries solves all of the issues previously raised with regard to anonymous namedtuples. References: [1] https://mail.python.org/pipermail/python-ideas/2016-April/039857.html [View Less]

10 17

Allow `return yield from`
by Patrick Reader March 29, 2022

March 29, 2022

(I originally wrote this in bpo47147, but apparently because it has to go through python-ideas first?) I would like to be able to use a `yield from` expression in a `return` statement without parentheses, as a small quality of life tweak, i.e.: return yield from gen instead of return (yield from gen) I think this makes sense, since `yield from` can be used on the right-hand-side of an assignment, which accepts any expression, and so should `return`. Here is a medium-sized real-world … [View More]

3 3

Add __match_args__ to complex and slice
by Patrick Reader March 28, 2022

March 28, 2022

I'd like to be able to deconstruct `complex` objects into their real and imaginary components using a `match` statement, like: match 1-4j: case complex(a, b): print(f"{a=} {b=}") This would just require setting complex.__match_args__ = ("real", "imag") The other builtin type I think should be able to do this is `slice`: … [View More]

2 1

Regex timeouts
by J.B. Langston March 22, 2022

March 22, 2022

Hello, I had opened this bug because I had a bad regex in my code that was causing python to hang in the regex evaluation: https://bugs.python.org/issue46627. I have fixed the problem with that specific regex, but more generally I think it would be good to have a timeout option that could be configurable when compiling the regex so that if the regex didn't complete within the specified timeframe, it would abort and throw an exception. The bug was closed as Won't Fix and it was suggested that … [View More]

14 50

A modulo operator consistent with euclidean division.
by Nathan Levett March 20, 2022

March 20, 2022

Howdy python gang, First time posting here ~ I've recently encountered that python does not have an OOTB operator for modulo that is consistent with Euclidean division. Although it should be easy for anyone who wants this to create it themselves, it struck me as odd that it was not an already included feature. I was even more shocked to see a list indicating that most languages don't include one consister with Euclidean division (disheartening to realise that the number theory way of doing … [View More]

11 17

Re: Add a replace method to tuples
by Steven D'Aprano March 17, 2022

March 17, 2022

On Tue, Mar 15, 2022 at 11:16:04AM +1100, Chris Angelico wrote: > > How do we make a new list with a change other than the same slice and > > concatenation we use with tuples? > > > > alist[:i] + [value] + alist[i+1:] > > > > I mean as an expression, of course we can split it over two statements: > > > > newlist = alist[:] > > newlist[i] = value > > > > which is fine, but it does lose the expressiveness of an … [View More]

5 7

Expand type coverage in cpython implementation
by Jared Crawford March 16, 2022

March 16, 2022

Hi all, I recently noticed that stdlib functions often don't have type hints. Is there a technical reason why typing isn't more widespread in the cpython implementation? As a developer, it'd be great if I didn't have to wrap many of my stdlib calls in casts to make mypy happy. Please let me know if there's a more appropriate channel for this discussion. I tried looking through the mailing list archives and github issues, but I couldn't find the answer to this. Best, Jared

3 2

Re: Add a replace method to tuples
by wfdc March 16, 2022

March 16, 2022

> why haven't you used a list 1. A list is not immutable. 2. I don't want to modify the original sequence. ------- Original Message ------- On Friday, March 11th, 2022 at 4:07 PM, Marco Sulla <Marco.Sulla.Python(a)gmail.com> wrote: > On Fri, 11 Mar 2022 at 21:39, wfdc via Python-ideas > > python-ideas(a)python.org wrote: > > > For me specifically, my latest use-case is in a game-theoretic setting where the tuple represents a strategy profile. But the details of … [View More]

8 29

Add a replace method to tuples
by wfdc March 15, 2022

March 15, 2022

Add a "replace" method to tuples that returns a new tuple with the element at a given index replaced with a given value. Example implementation: def replace(self, index, value): return self[:index] + (value,) + self[index + 1:] See https://stackoverflow.com/questions/11458239/how-to-change-values-in-a-tuple for more context. Currently, tuples have 2 public methods: index and count. replace would be similarly easy to implement and similarly useful. Furthermore, it would be a natural … [View More]

12 56

hashlib.sha256(file_obj)
by Aur Saraf March 14, 2022

March 14, 2022

Hi, TL;DR: you can read just the code snippets and the last paragraph. First of all, I'm assuming hashlib is used to calculate hashes of large files in production very very often, such that even small performance and usability improvements would make a huge difference. If you don't share this assumption, delete this email :-) Today, hashlib.update() accepts only a bytes(), and many StackOverflow answers include code like: sha256 = hashlib.sha256() with open(path) as f: while True: … [View More]

1 1

X-Post: Adopting the concept of "Teams" (from PEP 8015)
by Pradyun Gedam March 14, 2022

March 14, 2022

This is basically an email asking for inputs on https://discuss.python.org/t/14210. It's not clear to me whether the expectation is that people post in two places or just one; so I'm going to avoid duplicating the contents here. TLDR from that proposal: PEP 8015 had proposed an SC + Core Developers + Teams model. Python's current governance model has the first two, while the third is something that we've informally organised into. Let's formalise the organisational structures we already have, … [View More]

1 0

Re: [Python-Dev] An unambiguous way of initializing an empty dictionary and set
by Ethan Furman March 13, 2022

March 13, 2022

On 3/13/22 14:49, joao.p.f.batista.97(a)gmail.com wrote: > Currently: > l = [] # new empty list > t = () # new empty tuple > s = set() # new empty set (no clean and consistent way of initializing regarding the others) <<< > d = {} # new empty dictionary > > Possible solution: > s = {} # new empty set > d = {:} # new empty dictionary (the ":" is a reference to key-value pairs) > > Current workaround at least for consistency: > l = list() #… [View More]

1 0

Proposal to associate thread-local data/context with a faulthandler traceback
by tom＠tomforb.es March 11, 2022

March 11, 2022

The faulthandler module is invaluable for tracking down segfaults in native code, however it is really lacking the ability to add some kind of useful breadcumb to aide debugging. Imagine you are running a large-scale distributed job over tens of millions of images and a single one causes opencv to segfault reliably. This can be very difficult to track down. It would be very useful to be able to add a string (limited by size) via the faulthandler module that is outputted along with the … [View More]

4 6

suprocess.check_call and friends include STDERR in error message
by mangelozzi＠gmail.com March 9, 2022

March 9, 2022

I often wish to perform the action of `subprocess.check_output`, i.e. print STDOUT live, and check the return code is 0 on exit. However if there is an error, I would like the STDERR to be printed. When looking at log files and just seeing the return code is often not very beneficial. E.g.: ```py subprocess.check_call(['ping', '-c', '1', 'github.com2']) ``` Will generate this error: ``` subprocess.CalledProcessError: Command '['ping', '-c', '1', 'github.com2']' returned non-zero exit status … [View More]

2 1

shutil.copyfileobj to return number of bytes copied
by Davis, Matthew March 6, 2022

March 6, 2022

Hi, Currently shutil.copyfileobj returns nothing. I would like to be able to find out how many bytes were copied. Whilst most file-like objects have a .tell() which you could use, some don’t, and .tell() is not guaranteed to measure the number of bytes, it could measure with other units. I don’t think changing the return value from None to not-None would be backwards incompatible. We could also do the same change for all other shutil.copy* methods (except copytree). Looking at the code, … [View More]

3 3

Add function for reading a specific number of bytes/characters from a file object that fails noisily
by Kevin Mills March 4, 2022

March 4, 2022

The code I'm currently working on involves parsing binary data. If I ask for, say, 4 bytes, it's because I actually need 4 bytes and if the file doesn't have 4 bytes for me, it's malformed. Because `f.read(4)` can silently return less than 4 bytes and I don't want to have to explicitly double check every read, I'm using a wrapper function. def read_exact(f, size): data = f.read(size) if len(data) < size: raise EOFError(f"expected read of size {size}, got {len(data)}") … [View More]

5 8