Mailman 3 October 2021 - Python-ideas

Type-hinting dictionaries for an arbitrary number of arbitrary key/value pairs? Counterpart to PEP 589?
by Sebastian M. Ernst 16 Oct '21

16 Oct '21

Hi all, disclaimer: I have no idea on potential syntax or if it is applicable to a wide audience or if there is already a good solution to this. It is more like a "gap" in the type hint spec that I ran across in a project. In function/method signatures, I can hint at dictionaries for example as follows: ```python from numbers import Number from typing import Dict from typeguard import typechecked Data = Dict[str, Number] @typechecked def foo(bar: Data): print(bar) ``` Yes, this is using run-time checks (typeguard), which works just fine. Only strings as keys and Number objects as values are going through. (I was told that MyPy does not get this at the moment.) The issue is that `bar` itself still allows "everything" to go in (and out): ```python @typechecked def foo2(bar: Data): bar[1.0] = b'should not be allowed' ``` PEP 589 introduces typed dictionaries, but for a fixed set of predefined keys (similar to struct-like constructs in other languages). In contrast, I am looking for an arbitrary number of typed keys/value pairs. For reference, related question on SO: https://stackoverflow.com/q/69555006/1672565 Best regards, Sebastian

6 8

Antichecked exceptions
by Soni L. 16 Oct '21

16 Oct '21

PEP 479 introduced this idea that basically boils down to raising your exceptions only when you mean it. Specifically: generators only raise StopIteration when they return/exit/terminate. And they prevent the user from raising it under other circumstances. Can we call this "antichecked exceptions"? Generators specifically check for StopIteration raised in undesired circumstances, and make sure they get wrapped. This is a form of runtime checking, so the "checked" part of the name makes sense. But this is unlike "checked exceptions", like those in java, that mostly just encourage uhh silently ignoring the exception? (And those are *statically* checked anyway...) So the prefix "anti" distinguishes it from that. Obviously this name is meant to be more broad than just the implementation of generators. (We made a pypi package that tries to expand on the concept.) But, thoughts? Naming things is powerful. (Is this the wrong list for this? Uhh like it's a name idea for a thing in python? But uhh... it's not so much a change proposal? Except to refer to it by this new name? But anyway...)

2 1

Structure Pattern for annotation
by Abdulla Al Kathiri 15 Oct '21

15 Oct '21

Hello, Today I found myself write a function that returns a tuple of list of list of strings (tuple[list[list[str]], list[list[str]]]). Wouldn’t it easier to read to write it like the following: ([[str]], [[str]])? Similarly for TypedDict, replace the following.. class Movie(TypedDict): name: str year: int with {‘name’: str, ‘year’: int} dict[str, int] will be {str: int} set[int] will be {int}. This is similar to the new proposed Callable alternative (i.e., (int, float) -> float) since it mimics the structure of a function. Also, Annotated[float, “seconds”] can be replaced with something like float #seconds indicating whatever comes after the hashtag is just extra information similar to a normal comment in the code. Imagine you could replace the following def func(d: dict[Annotated[str, “product label”], Annotated[list[float], “prices from 2000 to 2015”]])… With def func(d: {str # product label: [float] # prices from 2000 to 2015})… Abdulla Sent from my iPhone

3 3

Re: dict_items.__getitem__?
by Alex Waygood 14 Oct '21

14 Oct '21

I think there definitely should be a more obvious way to do this (specifically the first and last keys/values/items of a dictionary — I'm ambivalent about the others, since they won't always be fast, as discussed). An anti-pattern you see quite often on Stack Overflow to get the first key of a dictionary is something like the following: first_key = list(mydict.keys())[0] Whereas obviously, a much better way (especially if it's a very large dictionary) is to do: first_key = next(iter(mydict)) [Christopher Barker] > I'll leave it exercise for the reader to find that thead For reference, the (very long) previous thread is here: https://mail.python.org/archives/list/python-ideas@python.org/thread/S7UMTW…. [Inada Naoki] > I think we can add `itertools.first()` for this idiom, and > `itertools.last()` for `next(iter(reversed(x)))` idiom. I like this idea, a lot. Another possibility I've been wondering about was whether several methods should be added to the dict interface: dict.first_key = lambda self: next(iter(self)) dict.first_val = lambda self: next(iter(self.values())) dict.first_item = lambda self: next(iter(self.items())) dict.last_key = lambda self: next(reversed(self)) dict.last_val = lambda self: next(reversed(self.values())) dict.last_item = lambda self: next(reversed(self.items())) But I think I like a lot more the idea of adding general ways of doing these things to itertools. Best, Alex > On 5 Oct 2021, at 05:30, Christopher Barker <pythonchb(a)gmail.com> wrote: > > On Mon, Oct 4, 2021 at 5:46 PM Erik Demaine <edemaine(a)mit.edu> wrote: >> Have folks thought about allowing indexing dictionary views as in the >> following code, where d is a dict object? >> >> d.keys()[0] >> d.keys()[-1] >> d.values()[0] >> d.values()[-1] >> d.items()[0] >> d.items()[-1] # item that would be returned by d.popitem() > > since dicts were made order-preserving, indexing the keys, items, etc does make some sense. > > > > I've also often wanted to get an arbitrary item/key from a dictionary, and > > This is indeed one of the use cases identified. > >> I found some related discussion in >> https://mail.python.org/archives/list/python-ideas@python.org/thread/QVTGZD… >> but not this exact idea. > > That's a pretty different idea but this exact idea has been discussed on this list relatively recently. I still like it, but there wan't much general support. > > I'll leave it exercise for the read to find that thead, but it is there, and I suggest you look for it if you want to further pursue this idea. > > -CHB > > > -- > Christopher Barker, PhD (Chris) > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RAEDZ… > Code of Conduct: http://python.org/psf/codeofconduct/

18 63

Overloading unary plus in strings with "ord"
by Jeremiah Vivian 12 Oct '21

12 Oct '21

So `ord` is already a really fast function with (last check before this thread was posted) 166 nsec per loop. But I'm wondering... doing `ord(a)` produces this bytecode: > 2 4 LOAD_NAME 1 (ord) > 6 LOAD_NAME 0 (a) > 8 CALL_FUNCTION 1 > 10 POP_TOP > 12 LOAD_CONST 1 (None) > 14 RETURN_VALUE But doing `+a` only produces this: > 2 4 LOAD_NAME 0 (a) > 6 UNARY_POSITIVE > 8 POP_TOP > 10 LOAD_CONST 1 (None) > 12 RETURN_VALUE So an operator has its own bytecode, doesn't need to `LOAD_*` a function, and doesn't have the impact on performance when converting arguments to the format of the (after `POP()`ing every argument) TOS function and then calling that function. And also, the unary `+` of strings only copies strings, which should be redundant in most cases. Maybe `ord` can take the place of unary `+` in strings?

5 18

Adding a Version type
by Finn Mason 11 Oct '21

11 Oct '21

Hello all, I was thinking about the following idioms: __version__ = "1.0" import sys if sys.version_info < (3, 6): # Yell at the user I feel like there could be a better way to define versions. There's no real standard way to specify versions in Python, other than "use semantic versioning." This can cause problems such as: * Uncertainty of the type of the version (commence the pattern matching! Or `if... elif` statements if that's your thing) * No real way to specify an alpha, beta, pre, or post version number in a tuple of ints, e.g. `(2, 1, 0, "post0")` doesn't really work. I propose a new `Version` type, a standard way to define versions. The signature would look something like `Version(major: int, minor: int = 0, patch: Optional[int] = None, alpha: Optional[int] = None, beta: Optional[int] = None, pre: Optional[int] = None, post: Optional[int] = None)`. To maintain backwards compatibility, comparisons such as `Version(1, 2) == (1, 2)` and `Version(1, 2) == "1.2"` will return `True`. `str(Version(1, 2))` will return `"1.2"` (to clarify, if `patch` were 0 instead of None, it would return `"1.2.0"`). There will be an `as_tuple()` method to give the version number as a tuple (maybe named tuple?), e.g. `Version(1, 2).as_tuple()` will return `(1, 2)`. A problem is that not all versioning systems are covered by this proposal. For example, I've done some programming in C#, and many C# programs use a four number system instead of the three number system usually used in Python code, i.e. major.minor.patch.bugfix instead of major.minor.patch. (This may not seem very different, but it often is.) Where to place this type is an open question. I honestly have no good ideas. This could just be implemented as a PyPI package, but there may not be a point in adding a whole dependency to use a semantic type that other people probably aren't using (which wouldn't be the case if it were part of core Python). This would be implemented in Python 3.11 or 3.12, but I'd recommend a backport on PyPI if implemented. Questions? Thoughts? Suggestions to improve it?

6 8

socketserver StreamRequestHandler non-blocking read__
by Evan Greenup 10 Oct '21

10 Oct '21

For StreamRequetHandler's rfile attribute, it will block when the read readline readlines reach the end. It would be nice to add non-blocking read because the seek() is unavailable for rfile. and there is no other way to detect if it is ended. When the stream is empty and waiting for new input, then it will return None type. This would be a nice feature.

1 0

There must be a better way to do class-attributes -- allow descriptor.__set__ on `type`, not just `object`
by Randolf Scholz 10 Oct '21

10 Oct '21

Over the past few weeks/months I have had on an off issues with the way python handles class-attributes. Consider the default way to do it: ``` class A: attr = 2 """Docstring of the attribute""" ``` There are 2 main issues I have with this way of setting class-attributes: 1. It does not allow to set class-attributes that depend on other class-attributes. (Example: setting a path that depends on cls.__name__) 2. It does not allow to set lazily evaluated attributes. If setting the class-attribute involves an expensive computation, it will be done at class definition time. One possible resolution is to use metaclasses, however this comes with its own issues: 1. MetaClasses do not work nice with documentation. Attributes defined my metaclasses will neither show up in help(), dir(), nor will they be in the correct place when documenting the class with tools like sphinx. 2. Who wants to write an additional metaclass whenever they want to write a class and have some minimal class-attribute processing? A new possible resolution, possible since python 3.9, is the combination of @classmethod and @property. However, this has it's own problems and pitfalls, cf. https://bugs.python.org/issue44904, https://bugs.python.org/issue45356. In fact, after lots of trial and error and reading into descriptors I figured that "true" class-properties are not possible in the current versions of python for a very simple reason: ╔══════════════════════════════════════════════════════╗ ║descriptor's __set__ method is never called when setting a class-attribute. (see MWE) ║ ╚══════════════════════════════════════════════════════╝ Therefore, I propose the following: 1. By default `type.__setattr__ ` should check whether the attribute implements `__set__` and if so, use that instead. (it already works like this for object.__setattr__), (cf. PatchedTrig below) 2. There should be a clearer distinction between class-attributes and instances-attributes. Instances should probably by default not be allowed to change class-attributes, since this can affect global state. 3. It would be very nice to have a built-in @attribute decorator, much like @property, but specifically for stetting class attributes. (again, metaclasses are possible but the issues with documentation compatibility seem very bad) (pretty much what @classmethod@property promises but doesn't fulfill.) ## MWE ```python import math class TrigConst: def __init__(self, const=math.pi): self.const = const def __get__(self, obj, objtype=None): return self.const def __set__(self, obj, value): print(f"__set__ called", f"{obj=}") self.const = value class Trig: const = TrigConst() ``` Then ```python Trig().const # calls TrigConst.__get__ Trig().const = math.tau # calls TrigConst.__set__ Trig.const # calls TrigConst.__get__ Trig.const = math.pi # overwrites TrigConst attribute with float. ``` Patched version ```python class PatchedSetattr(type): def __setattr__(cls, key, value): if hasattr(cls, key): obj = cls.__dict__[key] if hasattr(obj, "__set__"): obj.__set__(cls, value) else: super().__setattr__(key, value) class PatchedTrig(metaclass=PatchedSetattr): const = TrigConst() ``` Then ```python cls = Trig print(cls.const, type(cls.__dict__["const"])) cls.const = math.tau print(cls.const, type(cls.__dict__["const"])) ``` gives ``` 3.141592653589793 <class '__main__.TrigConst'> 6.283185307179586 <class 'float'> ``` But ```cls cls = PatchedTrig print(cls.const, type(cls.__dict__["const"])) cls.const = math.tau print(cls.const, type(cls.__dict__["const"])) ``` gives ``` 3.141592653589793 <class '__main__.TrigConst'> __set__ called obj=<class '__main__.PatchedTrig'> 6.283185307179586 <class '__main__.TrigConst'> ```

1 0

Invalidate list iterators after size changes?
by Tyler Hou 09 Oct '21

09 Oct '21

Right now, the following code is valid in Python 3.9 (and infinitely loops): ``` lst = [1, 2, 3] for i in lst: lst.append(i) ``` IMO this is a source of bugs, and even though this behavior is technically documented (https://bugs.python.org/issue32767), I don't know if it should belong in the core language. For example, you cannot change the size of a set or dict during iteration; you will get a RuntimeError. Allegedly this is due to the underlying implementation of the dict/set iterators (https://bugs.python.org/issue22084) but I think it would be beneficial this was made part of the public API for list, set, and dict iterators. Concretely, I'm proposing the following set of changes to Python: 1. If the size of a list, set, or dict changes, invalidate all existing iterators to that containers. Document this behavior as part of the public API. 2. Attempting to call next() on an invalidated iterator raises a RuntimeError. 3. Add a key_iter function to itertools (name up for debate), which replicates the existing functionality for lists. It could also make a copy of the keys of a set/dict, so you could use it to mutate those as well during iteration (to allow users to write functions that mutate an arbitrary duck-typed collection) 4. Gate (1, 2) for lists behind a __future__ flag with a long migration window (e.g. ~3.16). Benefits: 1. Reduce a source of bugs (deleting while iterating forward through a list often means that your logic skips elements; appending can lead to infinite loops or unexpected behavior). 2. Make the iterator invalidation consistent between lists, sets, and dicts. 3. More consistent behavior with collections in other languages (C++, Java). Drawbacks: 1. Migration costs. But mutating during iteration should be relatively rare, given that most people discourage mutating while iterating (https://google.github.io/styleguide/pyguide.html#284-decision, https://stackoverflow.com/a/1637875/1625484) __future__ feature flag plus deprecation warnings should go along way to help people migrate their code. We would also offer an easy in-place fix (key_iter). 2. You can't write algorithms that append to a list (e.g. BFS) with the normal list iterator. Again, you could use key_iter (or use a deque with block size == 1; see Open Questions). Open Questions Is (3) necessary? It's not hard to write an implementation yourself. What should be done about deque for block sizes > 1? If this change gets made, should we add way to create a deque with block size == 1 whose iterators are never invalidated?

6 5

String method to check for a float
by dpalit17＠outlook.com 07 Oct '21

07 Oct '21

It would be nice to have a string method that checks for a float. Currently there is no support for this, either built-in or in the standard library. There is a thread, dating back to Dec 2020, that proposes a trivial implementation for str.isfloat . I was thinking of a method that did more. Consider the following code. It returns True if the string is a proper float, False if it is an int and None otherwise. def isfloat(s): try: int(s) return False except ValueError: try: float(s) return True except ValueError: return None This will be useful when we want to preserve the type of the number that is in string format. Anywhere a number is input as a string to a method and we want to later on output the original number, we can use the above. If, instead (as suggested in the other thread), the string is simply converted to a float, then the info that the string was an int is lost.

6 20