Mailman 3 March 2022 - Python-ideas

Allow `return yield from`
by Patrick Reader 29 Mar '22

29 Mar '22

(I originally wrote this in bpo47147, but apparently because it has to go through python-ideas first?) I would like to be able to use a `yield from` expression in a `return` statement without parentheses, as a small quality of life tweak, i.e.: return yield from gen instead of return (yield from gen) I think this makes sense, since `yield from` can be used on the right-hand-side of an assignment, which accepts any expression, and so should `return`. Here is a medium-sized real-world example of where I'm using this, where it would be nice to allow `return yield from`:https://gist.github.com/pxeger/48f97484364bf0b43dee974a8f0f4265 Patrick

3 3

Add __match_args__ to complex and slice
by Patrick Reader 28 Mar '22

28 Mar '22

I'd like to be able to deconstruct `complex` objects into their real and imaginary components using a `match` statement, like: match 1-4j: case complex(a, b): print(f"{a=} {b=}") This would just require setting complex.__match_args__ = ("real", "imag") The other builtin type I think should be able to do this is `slice`: match slice(1, -1, None): case slice(start, stop, step): print(f"{start=} {stop=} {step=}") which would require slice.__match_args__ = ("start", "stop", "step") This would be useful, for example, in custom __getattr__ methods which need to handle slices. def __getattr__(self, arg): match arg: case slice(start, stop, end): ... # handle slice case n if isinstance(n, int): ... # handle single index case _: raise TypeError There are various other builtin types which could conceivably have __match_args__ defined, but with varying degrees of usefulness: range, property, staticmethod, classmethod, memoryview, types.GenericAlias, types.UnionType, types.CodeType, types.FunctionType, types.MethodType, types.ModuleType, types.GenericAlias, types.FrameType, types.TracebackType, types.CellType But none of these are really that important, I think. What do you think? Patrick

2 1

Regex timeouts
by J.B. Langston 21 Mar '22

21 Mar '22

Hello, I had opened this bug because I had a bad regex in my code that was causing python to hang in the regex evaluation: https://bugs.python.org/issue46627. I have fixed the problem with that specific regex, but more generally I think it would be good to have a timeout option that could be configurable when compiling the regex so that if the regex didn't complete within the specified timeframe, it would abort and throw an exception. The bug was closed as Won't Fix and it was suggested that I bring up my idea here. The suggestion made to me on the bug was to read Mastering Regular Expressions and get better at writing regexes. I will take this advice, but this isn't really a reasonable solution to my problem for a few reasons. My use case is log parsing and I have a large number of regexes that run over many different log lines. With the volume of regexes I have, it's hard to make sure every regex has no potential problems, especially when the pathological behavior only occurs on certain inputs that may not have been anticipated when developing the regex. Also because of the volume of data these regexes are parsing, I would never want to allow a regex to run longer than a few milliseconds because if it did, that would kill my log processing throughput. I'd rather that it just raise an exception and move on to the next log entry. Thanks, J.B.

14 50

A modulo operator consistent with euclidean division.
by Nathan Levett 20 Mar '22

20 Mar '22

Howdy python gang, First time posting here ~ I've recently encountered that python does not have an OOTB operator for modulo that is consistent with Euclidean division. Although it should be easy for anyone who wants this to create it themselves, it struck me as odd that it was not an already included feature. I was even more shocked to see a list indicating that most languages don't include one consister with Euclidean division (disheartening to realise that the number theory way of doing things is not the default). I was so shocked at it's lack from python that it motivated me to post this, I suppose! I guess I'm posting to check how open anyone would be to the idea? I'm not sure if '%%' is defined anywhere, but it seemed like an intuitive suggestion if not already used as an operator, parallel to the syntax of the '**' operator. Keen to know how open y'all're to it!

11 17

Re: Add a replace method to tuples
by Steven D'Aprano 17 Mar '22

17 Mar '22

On Tue, Mar 15, 2022 at 11:16:04AM +1100, Chris Angelico wrote: > > How do we make a new list with a change other than the same slice and > > concatenation we use with tuples? > > > > alist[:i] + [value] + alist[i+1:] > > > > I mean as an expression, of course we can split it over two statements: > > > > newlist = alist[:] > > newlist[i] = value > > > > which is fine, but it does lose the expressiveness of an expression :-) > > > > The functional programming style would be mapping most elements to > themselves, and the one you're replacing to the replacement. No, that doesn't work, because that is replacing by value, not by position. You're answering the wrong problem. > (Or, if you need to do it with indices, map the range of integers to > either the original element at that index or the replacement.) You mean this? [value if index == i else alist[i] for index in range(len(alist))] This would be *slightly* less unpythonic: [value if index == i else x for (index, x) in enumerate(alist)] but I would reject both of those in a code review unless you had clear proof that they were significantly faster or more optimal than the standard idioms. Which I am confident you won't have. I'm so confident that they're slower I'm not even going to time it myself before saying I'm confident they're slower! *wink* Famous-last-words-ly y'rs, -- Steve

5 7

Expand type coverage in cpython implementation
by Jared Crawford 16 Mar '22

16 Mar '22

Hi all, I recently noticed that stdlib functions often don't have type hints. Is there a technical reason why typing isn't more widespread in the cpython implementation? As a developer, it'd be great if I didn't have to wrap many of my stdlib calls in casts to make mypy happy. Please let me know if there's a more appropriate channel for this discussion. I tried looking through the mailing list archives and github issues, but I couldn't find the answer to this. Best, Jared

3 2

Re: Add a replace method to tuples
by wfdc 16 Mar '22

16 Mar '22

> why haven't you used a list 1. A list is not immutable. 2. I don't want to modify the original sequence. ------- Original Message ------- On Friday, March 11th, 2022 at 4:07 PM, Marco Sulla <Marco.Sulla.Python(a)gmail.com> wrote: > On Fri, 11 Mar 2022 at 21:39, wfdc via Python-ideas > > python-ideas(a)python.org wrote: > > > For me specifically, my latest use-case is in a game-theoretic setting where the tuple represents a strategy profile. But the details of that would take us far beyond the scope of the discussion. > > Well, but I think this is the point. I mean, why haven't you used a > > list? I suppose the main doubt for me and the others is here. > > I mean, I like your proposal (but this probably is not helping you > > here...), and I'm somewhat attracted by functional programming. But I > > never programmed in functional style. It's more a sympathy and a > > fascination than a real need. But maybe you can change my mind with > > your personal experience, as I changed my opinion about OOP a long > > time ago.

8 29

Add a replace method to tuples
by wfdc 15 Mar '22

15 Mar '22

Add a "replace" method to tuples that returns a new tuple with the element at a given index replaced with a given value. Example implementation: def replace(self, index, value): return self[:index] + (value,) + self[index + 1:] See https://stackoverflow.com/questions/11458239/how-to-change-values-in-a-tuple for more context. Currently, tuples have 2 public methods: index and count. replace would be similarly easy to implement and similarly useful. Furthermore, it would be a natural counterpart to nametuple's _replace method.

12 56

hashlib.sha256(file_obj)
by Aur Saraf 14 Mar '22

14 Mar '22

Hi, TL;DR: you can read just the code snippets and the last paragraph. First of all, I'm assuming hashlib is used to calculate hashes of large files in production very very often, such that even small performance and usability improvements would make a huge difference. If you don't share this assumption, delete this email :-) Today, hashlib.update() accepts only a bytes(), and many StackOverflow answers include code like: sha256 = hashlib.sha256() with open(path) as f: while True: data = f.read(BUF_SIZE) if not data: break sha256.update(data) return sha256.hexdigest() This is problematic because not everybody knows this pattern, so many codebases will include code like: with open(path) as f: return sha256(f.read()).hexdigest() and, frankly, who can blame them. It is also problematic for performance reasons - even if we know to do the chunking thing, while hashing a large file, the GIL will be taken and released many times, and many buffers will be allocated and deallocated. As far as I can see, hashlib already has a lock per hash object and safely releases the GIL in update() with a long bytes(), so it would be safe to add an option for update()/new() to take a file pointer and do the chunked reading/updating with one static buffer with the GIL released throughout, so that this would be with open(path) as f: return sha256(f).hexdigest() We can discuss whether this is the best API or its preferable to have with open(path) as f: return sha256.from_file(f).hexdigest() or with open(path) as f: return sha256().update_from_file(f).hexdigest() but I submit that today many people try sha256(f).hexdigest() because they're used to e.g. json and csv accepting file objects, and that today passing a file object raises, so making both new() and update() accept file objects would be the most beginner-friendly and won't break anything. Knowing that there are a million tiny details that need to be... hashed out, and given that I'm willing to write the code, would the devs be receptive to something like this? Thanks, Aur

1 1

X-Post: Adopting the concept of "Teams" (from PEP 8015)
by Pradyun Gedam 14 Mar '22

14 Mar '22

This is basically an email asking for inputs on https://discuss.python.org/t/14210. It's not clear to me whether the expectation is that people post in two places or just one; so I'm going to avoid duplicating the contents here. TLDR from that proposal: PEP 8015 had proposed an SC + Core Developers + Teams model. Python's current governance model has the first two, while the third is something that we've informally organised into. Let's formalise the organisational structures we already have, change the names of various groups to "teams" and benefit from the clearer formal organisation + naming scheme.

1 0

Python-ideas March 2022