Mailman 3 May 2020 - Python-ideas

Python tagged unions based on dataclasses
by Andrey Cizov 23 May '20

23 May '20

I have developed a library to introduce tagged unions to python that uses dataclasses to define disjoint members of the tagged union (by defining them as Optional fields). With some additional validation I make sure that only one of the fields is not None. I find that it also fits well with the existing analysis tools and IDEs (e.g. PyCharm) and doesn’t require any additional work in order to be supported. I would like to see some criticism and whether that could potentially be a good candidate for python standard library in the future. -- Andrey Cizov

3 5

Enhance list.index with a comparison function
by Steven D'Aprano 23 May '20

23 May '20

Have a look at this short thread on discuss: https://discuss.python.org/t/searching-array-of-data-objects/4251/1 After I answered that question, it dawned on me that I have probably written something like that loop, or variations of it, a thousand times: for obj in somelist: if comparison(obj, needle): do_something(obj) break and now it's a thousand and one times *wink* So here's a thought... suppose we gave list.index an optional keyword- only comparison function? The typical name for those is "pred", as in predicate. Then the above becomes: try: i = somelist.index(needle, pred=comparison) except ValueError: pass else: do_something(somelist[i]) If you know for sure the search item exists, then you can drop the try...except...else: do_something(somelist[somelist.index(needle, pred=comparison)]) If `pred` is missing or None, a standard equality search is performed. Together with the operator module, this would allow us to perform common searches at C speed: # find the first item less than needle somelist.index(needle, pred=operator.lt) We can search a sublist efficiently: # copies the sublist :-( for obj in somelist[25:95354]: ... # save memory by inefficiently walking the entire list :-( for i, obj in enumerate(somelist): if i < 25: continue if i >= 95354: break ... # avoids both wasting memory and wasting time :-) i = somelist.index(needle, 25, 95354, pred=comparison) What do you think? -- Steven

7 11

Re: [Python-Dev] Re: PEP 618: Add Optional Length-Checking To zip
by Steven D'Aprano 21 May '20

21 May '20

On Fri, May 15, 2020 at 05:46:43PM +0200, Antoine Pitrou wrote: > On Fri, 15 May 2020 10:46:25 -0400 > David Mertz <mertz(a)gnosis.cx> wrote: > > > > > 1. +1 itertools.zip_strict function > > > 2. +1 zip.strict(*args) > > > 3. +1 zip(*args, mode='strict') # mode='shortest' by default > > > 4. +0 zip(*args, strict=True) > > > > > > > Mostly I agree with Steven on relative preference: > > > > itertools.zip_strict() +1 > > zip.strict() +0.5 > > zip(mode='strict') +0 > > zip(strict=True) -0.5 > > For me: > > * zip(strict=True) +1 > * zip(mode='strict') -0 > * itertools.zip_strict() -0.5 > * zip.strict() -1 (but really, I'd like to make this -1e10) I spent a significant amount of time and mental energy explaining in detail why a boolean flag is a poor API and is objectively the wrong interface here. This is not just a matter of personal taste: there are reasons why a flag is wrong here. Flags are for combining independent and orthogonal settings which can be combined. This is not such a feature. I think it is objectively the worst API on the table here, for reasons already discussed. - Your preferred option makes the strict zip version a second-class citizen of the language; - your preferred option is the least open to future enhancements; - your most hated option is the one which follows the Zen of Python the most closely (namely, the koan about having more namespaces); - and is the most object-oriented solution (it's effectively a method); - and most importantly you explicitly *oppose* every alternative API, giving them negative preferences; you would rather not have this zip variant at all than have an interface other than a boolean flag. If I needed this function[1], I'd accept it even if it were spelled `xyghasx.peyahc.flihaj()` and required me to set a global variable to make it work. So I can't help but interpret your total opposition to every other interface as a strong sign that you don't really need this zip variant at all. If your opinion is typical, perhaps we should just reject the PEP. Could you explain why you believe a bool flag is the only suitable interface? Objective reasons preferred please. The same goes for everyone else who gave a vote. If you have an objective reason for wanting this strict zip function to be a second class citizen that cannot be easily extended in the future, then please explain why. [1] I don't think I do. If I had my preference, thinking only of my own needs, I'd probably reject the PEP. But I acknowledge that others seem to need it. -- Steven

10 20

PEP 618: Add Optional Length-Checking To zip
by Brandt Bucher 20 May '20

20 May '20

I have pushed a first draft of PEP 618: https://www.python.org/dev/peps/pep-0618 Please let me know what you think – I'd love to hear any *new* feedback that hasn't yet been addressed in the PEP! Brandt

26 72

str.strip: argument maxstrip
by computermaster360 . 20 May '20

20 May '20

I often find myself in a need of stripping only a specified amount of characters from a string (most commonly a single character). I have been implementing this in an ad-hoc manner, which is quite inelegant: def rstrip1(txt, chars): if txt is None: return None elif any(txt.endswith(c) for c in chars): return txt[:-1] else: return txt I would appreciate if str.split, str.lstrip, str.rsplit functions had a `maxstrip` argument, similar to the `maxsplit` argument of `str.split`, which would specify the maximum count of characters to be removed from the string. In case of `str.split` this maximum would apply to each side of the string separately. Am I the only one who is surprised such functionality is not implemented already??

7 17

Ensuring asserts are always on in a file
by Alex Hall 19 May '20

19 May '20

Some people (like myself, or the coworkers of [this person](https://mail.python.org/archives/list/python-ideas@python.org/threa…) just like to use asserts as a convenient way to check things. We don't want asserts to ever be turned off. Maybe there could be some kind of compiler directive which means "in this file, even with -O, keep the asserts". Maybe the line `assert __debug__`?

5 7

Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()
by remi.lapeyre＠henki.fr 18 May '20

18 May '20

Both urlretrieve() and urlcleanup() were kept for backward compatibility with Python2 but they were never deprecated like urllib.request.URLOpener and urllib.request.FancyURLOpener. Now that Python2 has been definitely sunset, can we raise a deprecation warning so that they can be removed in the future?

2 4

islice with actual slices
by Ram Rachum 17 May '20

17 May '20

Here's an idea I've had. How about instead of this: itertools.islice(iterable, 7, 20) We'll just have: itertools.islice(iterable)[7:20] Advantages: 1. More familiar slicing syntax. 2. No need to awkwardly use None when you're interested in just specifying the end of the slice without specifying the start, i.e. islic(x)[:10] instead of islice(x, None, 10) 3. Doesn't require breaking backwards compatibility. What do you think?

13 79

Re: Auto-assign attributes from __init__ arguments
by Alex Hall 17 May '20

17 May '20

Steven D'Aprano wrote: > Proposal: > We should have a mechanism that collects the current function or > method's parameters into a dict, similar to the way locals() returns all > local variables. > This mechanism could be a new function,or it could even be a magic local > variable inside each function, similar to what is done to make super() > work. But for the sake of this discussion, I'll assume it is a function, > parameters(), without worrying about whether it is a built-in or > imported from the inspect module. > So given a function signature: > def func(spam, eggs, cheese=None): > > and a call to it func(1, 2), then inside the body of the function, > calling parameters() will return the dict: > {'spam': 1, 'eggs': 2, 'cheese': None} > [...] > Another pattern would be to pass on your parameters to the superclasses: > super().method(**parameters()) I took a stab at this. This should work regardless of where the function is defined. ``` import inspect def parameters(*, expand_kwargs=True): frame = inspect.currentframe().f_back code = frame.f_code varnames = code.co_varnames end = code.co_argcount + code.co_kwonlyargcount result = { name: frame.f_locals[name] for name in varnames[:end] } # **kwargs if code.co_flags & inspect.CO_VARKEYWORDS: index = end if code.co_flags & inspect.CO_VARARGS: index += 1 name = varnames[index] var = frame.f_locals[name] if expand_kwargs: result.update(var) else: result[var] = name return result def foo1(a, b=1, *args, c, **d): x = 1 y = 2 print(parameters()) print(a, b, c, d) foo2(**parameters()) def foo2(a, b=1, *args, c, **d): print(a, b, c, d) foo1(1, 2, 3, 4, c=5, e=6, f=7) ``` Output: ``` {'a': 1, 'b': 2, 'c': 5, 'e': 6, 'f': 7} 1 2 5 {'e': 6, 'f': 7} 1 2 5 {'e': 6, 'f': 7} ``` Note that `parameters()` intentionally doesn't contain `'args': (3, 4)` because then `foo2(**parameters())` would see `args` as a keyword argument within `d` instead of bound to the parameter `*args`. So naturally I tried `foo2(*args, **parameters())` but that gave me `TypeError: foo2() got multiple values for argument 'a'`. So if you want to pass all arguments from one function to another, you need more than a dict if the inner signature include `*args`, and obviously even more so if it includes positional only parameters.

1 0

Auto-assign attributes from __init__ arguments
by Lewis Ball 16 May '20

16 May '20

Hi All, First of all, if this is something which has been discussed in the past the please point me in the right direction. *Problem:* When creating classes in Python, I find myself writing the __init__ method in a very similar way a lot of the time, that is: ``` def __init__(self, argument_1, argument_2, argument_3=None): self.argument_1 = argument_1 self.argument_2 = argument_2 self.argument_3 = argument_3 # then maybe some other attribute setting and logic follows ``` Every argument of __init__ gets a corresponding attribute with the same name. This means that each `argument_i` has been typed 3 times, which seems overly-verbose as well as being easy to mistype. This pattern is easy to find in various popular python libraries, and in some it is actually enforced. For example, I do quite a bit of work with classifiers using the sklearn estimator API, and for various reasons sklearn enforce this pattern for an __init__ (see here <https://scikit-learn.org/stable/developers/develop.html#instantiation> if interested). Here is an example of this pattern from the standard library (from textwrap.TextWrapper): ``` def __init__(self, width=70, initial_indent="", subsequent_indent="", expand_tabs=True, replace_whitespace=True, fix_sentence_endings=False, break_long_words=True, drop_whitespace=True, break_on_hyphens=True, tabsize=8, *, max_lines=None, placeholder=' [...]'): self.width = width self.initial_indent = initial_indent self.subsequent_indent = subsequent_indent self.expand_tabs = expand_tabs self.replace_whitespace = replace_whitespace self.fix_sentence_endings = fix_sentence_endings self.break_long_words = break_long_words self.drop_whitespace = drop_whitespace self.break_on_hyphens = break_on_hyphens self.tabsize = tabsize self.max_lines = max_lines self.placeholder = placeholder ``` With a quick scan of the top 50 or so most used python packages, *1 in 4* __init__ methods that takes arguments has the line `self.argument_i = argument_i` for every single argument, with several of them having 10+ arguments. *Suggestion:* A new built-in called something like `assign()` which would assign every single __init__ arg to a corresponding attribute. e.g. the snippet from above could be rewritten to: ``` def __init__(self, argument_1, argument_2, argument_3=None): assign() # other init logic goes here ``` This could alternatively be implemented as a decorator, like so ``` @assign def __init__(self, argument_1, argument_2, argument_3=None): # other init logic goes here ``` but then this requires a `pass` if no other logic is needed inside the __init__. There may also be some other syntax for this which would be even easier to use. Is this something that others would find useful? Thanks, Lewis

15 34