Mailman 3 September 2021 - Python-ideas

Re: Shorthand syntax for lambda functions that have a single parameter
by Jon Kiparsky 30 Sep '21

30 Sep '21

> Being too terse is worse than being too verbose This is my view as well. I would not want to see python go Perl-shaped.

1 0

PEP Draft: Power Assertion
by Noam Tenne 27 Sep '21

27 Sep '21

Hi All, Following the discussions on "Power Assertions: Is it PEP-able?", I've drafted this PEP. Your comments are most welcome. PEP: 9999 Title: Power Assertion Author: Noam Tenne <noam(a)10ne.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 24-Sep-2021 Abstract ======== This PEP introduces a language enhancement named "Power Assertion". The Power Assertion is inspired by a similar feature found in the `Groovy language`_ and is an extension to the core lib's ``assert`` keyword. When an assertion expression evaluates to ``False``, the output shows not only the failure, but also a breakdown of the evaluated expression from the inner part to the outer part. .. _Groovy language: http://docs.groovy-lang.org/next/html/documentation/core-testing-guide.html… Motivation ========= Every test boils down to the binary statement "Is this true or false?", whether you use the built-in assert keyword or a more advanced assertion method provided by a testing framework. When an assertion fails, the output is binary too — "Expected x, but got y". There are helpful libraries like Hamcrest which give you a more verbose breakdown of the difference and answer the question "What exactly is the difference between x and y?". This is extremely helpful, but it still focuses on the difference between the values. Keep in mind that a given state is normally an outcome of a series of states, that is, one outcome is a result of multiple conditions and causes. This is where Power Assertion comes in. It allows us to better understand what led to the failure. The Community Wants This ------------------------ As mentioned in the abstract, this feature was borrowed from Groovy. It is a very popular feature within the Groovy community, also used by projects such as `Spock`_. On top of that, it is very much needed in the Python community as well: * `Power Assertion was explicitly requested`_ as a feature in the `Nimoy`_ testing framework * There's a `similar feature in pytest`_ And when `discussed in the python-ideas`_ mailing list, the responses were overall positive: * "This is cool." - Guido van Rossum * "I was actually thinking exactly the opposite: this would more useful in production than in testing." - 2QdxY4RzWzUUiLuE＠potatochowder.com * "What pytest does is awesome. I though about implementing it in the standard compiler since seen it the first time." - Serhiy Storchaka .. _Spock: https://spockframework.org/ .. _Power Assertion was explicitly requested: https://stackoverflow.com/a/58536986/198825 .. _similar feature in pytest: https://docs.pytest.org/en/latest/how-to/assert.html .. _discussed in the python-ideas: https://mail.python.org/archives/list/python-ideas@python.org/thread/T26DR4… Rational ======== Code Example ------------ :: class SomeClass: def __init__(self): self.val = {'d': 'e'} def __str__(self): return str(self.val) sc = SomeClass() assert sc.val['d'] == 'f' This routine will result in the output: :: Assertion failed: sc.val['d'] == f | | | | e False | {'d': 'e'} Display ------- In the output above we can see the value of every part of the expression from left to right, mapped to their expression fragment with the pipe (``|``). The number of rows that are printed depend on the value of each fragment of the expression. If the value of a fragment is longer than the actual fragment (``{'d': 'e'}`` is longer than ``sc``), then the next value (``e``) will be printed on a new line which will appear above. Values are appended to the same line until it overflows in length to horizontal position of the next fragment. This way of presentation is clearer and more human friendly than the output offered by pytest's solution. The information that’s displayed is dictated by the type. If the type is a constant value, it will be displayed as is. If the type implements `__str__`, then the return value of that will be displayed. Mechanics --------- Reference Implementation '''''''''''''''''''''''' The reference implementation uses AST manipulation because this is the only way that this level of involvement can be achieved by a third party library. It iterates over every subexpression in the assert statement. Subexpressions are parts of the expression separated by a lookup (``map[key]``), an attribute reference (``key.value``) or a binary comparison operator (``==``). It then builds an AST in the structure of a tree to maintain the order of the operations in the original code, and tracks the original code of the subexpression together with the AST code of the subexpression and the original indices. It then rewrites the AST of the original expression to call a specialised assertion function, which accepts the tree as a parameter. At runtime the expression is executed. If it fails, a rendering function is called to render the assertion message as per the example above. Actual Implementation ''''''''''''''''''''' To be discussed. In the python-ideas mailing list, Serhiy Storchaka suggests: It needs a support in the compiler. The condition expression should be compiled to keep all immediate results of subexpressions on the stack. If the final result is true, immediate results are dropped. If it is false, the second argument of assert is evaluated and its value together with all immediate results of the first expression, together with references to corresponding subexpressions (as strings, ranges or AST nodes) are passed to the special handler. That handler can be implemented in a third-party library, because formatting and outputting a report is a complex task. The default handler can just raise an AttributeError. Caveats ------- It is important to note that expressions with side effects are affected by this feature. This is because in order to display this information, we must store references to the instances and not just the values. Reference Implementation ======================== There's a `complete implementation`_ of this enhancement in the `Nimoy`_ testing framework. It uses AST manipulation to remap the expression to a `data structure`_ at compile time, so that it can then be `evaluated and printed`_ at runtime. .. _Nimoy: https://browncoat-ninjas.github.io/nimoy/ .. _complete implementation: https://browncoat-ninjas.github.io/nimoy/examples/#power-assertions-beta .. _data structure: https://github.com/browncoat-ninjas/nimoy/blob/develop/nimoy/ast_tools/expr… .. _evaluated and printed: https://github.com/browncoat-ninjas/nimoy/blob/develop/nimoy/assertions/pow… Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

8 21

RFC: New/update the Docs Sphinx theme: responsive, edit page links,
by Wes Turner 22 Sep '21

22 Sep '21

Where would be the best place to discuss or just improve the CPython docs sphinx theme? What would Python need in a {PyData,} Sphinx theme fork: - [ ] ENH: add <edit on github> links to every docs page - [ ] ENH: responsive breakpoints for mobile devices - [ ] ENH: responsive fonts - [ ] ENH: toggle-able dark theme - [ ] ENH: interactive code examples (in WASM with pyodide like JupyterLite) - [ ] ENH?: Two sidebars: {Global TOC} {Content} {Document TOC} The PyData theme [bandwagon, feature justification, maintenance burden justification] Src: https://github.com/pydata/pydata-sphinx-theme Docs: https://pydata-sphinx-theme.readthedocs.io/en/latest/ ```quote Other sites that are using this theme: - Pandas: https://pandas.pydata.org/docs/ - NumPy: https://numpy.org/doc/stable/ - SciPy: https://scipy.github.io/devdocs/ - NetworkX: https://networkx.org/documentation/latest/ - Bokeh: https://docs.bokeh.org/en/latest/ - JupyterHub and Binder: https://docs.mybinder.org/, http://z2jh.jupyter.org/en/latest/, https://repo2docker.readthedocs.io/en/latest/, https://jupyterhub-team-compass.readthedocs.io/en/latest/ - Jupyter Book beta version uses an extension of this theme: https://beta.jupyterbook.org - Fairlearn: https://fairlearn.org/main/about/ ``` The sphinx-book-theme theme (Jupyter Book theme) has a few possibly necessary features for the {CPython, Devguide, *} docs: https://github.com/executablebooks/sphinx-book-theme https://sphinx-book-theme.readthedocs.io/en/latest/ :: ```quote A Sphinx theme with a clean design, support for interactive content, and a modern book-like look and feel. Flexible content layout: Inspired by beautiful online books, such as the Edward Tufte CSS guide Visual classes designed for Jupyter Notebooks: Cell inputs, outputs, and interactive functionality are all supported. Launch buttons for online interactivity: For pages that are built with computational material, connect your site to an online BinderHub for interactive content. Bootstrap 4: To style visual elements and add functionality. International: All text integrated in the theme is translated to the specified Sphinx language. ``` https://github.com/python/cpython/blob/main/Doc/conf.py#L68 : ```python html_theme = 'python_docs_theme' ``` https://github.com/python/cpython/blob/main/Doc/requirements.txt#L12 ``` # The theme used by the documentation is stored separately, so we need # to install that as well. python-docs-theme ``` Pypi: https://pypi.org/project/python-docs-theme/ Src: https://github.com/python/python-docs-theme/ What are some better ways to add {Source RST, Source, Edit, History, and Annotate} links to every sphinx doc page than sphinxcontrib-srclinks? https://github.com/westurner/sphinxcontrib-srclinks#usage - sphinxcontrib-srclinks: Source RST, Source, Edit, History, and Annotate, Clone URL - [ ] copy clone URL to clipboard button - pydata-theme: 'Edit this page' on the document TOC sidebar - sphinx-book-theme: {gh, bb, gl} icon > 'Suggest edit' - [ ] ENH: add collapsible sidebar to whatever is chosen as a suitable replacement for the current non-responsive theme

1 0

Typing Callable Ellipsis -- support for type hints a la Callable[[int, float, ...], None]
by Randolf Scholz 22 Sep '21

22 Sep '21

Dear Python community, type hinting is awesome! But recently I stumbled upon the following: I was writing a dictionary of functions that all share a common signature, in that the first two arguments were the same type. However, the functions could differ in the remaining arguments (positional / keyword) When trying to type hint as `Callable[[int, float, ...], None] `, which felt very natural at the time, mypy complained: `error: Unexpected "..."`. I propose to support type hints a la - Callable[[type1, type2, ..., typeN, ...], ReturnType] - Callable[[..., type1, type2, ..., typeN], ReturnType] which should match any callable with matching type hints for the first/last N inputs. (with some special treatment for functions with *args or **kwargs of course). Best regard, Randolf.

3 3

itertools.compress default selectors
by ml＠dyatkovskiy.com 22 Sep '21

22 Sep '21

Hi! I propose to enhance "itertools.compress" in such a way so if you don't provide selectors, then "data" itself is used as a selectors. So "compress(a)" would be equivalent to "compress(a, a)" For example: >>> from itertools import compress >>> [*compress([0, 1, 2, 3]))] [1, 2, 3] >>> [*compress(["", "CFLAGS=-O3"])] ["CFLAGS=-O3"] >>> opts = compress([None, "", "-filove-python", "CFLAGS=-O3"]) >>> " ".join(opts) '-filove-python CFLAGS=-O3' What do you think guys about this? Perhaps it was proposed by someone else? Thanks! Stepan Dyatkovskiy

7 13

Make localhost an acceptable parameter to ipaddress.IPv4Address address parameter
by Andres Torres 22 Sep '21

22 Sep '21

Title is pretty self explanatory. Right the following causes an error: ```py >>> ipaddress.IPv4Address('localhost') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\ProgramData\Anaconda3\lib\ipaddress.py", line 1252, in __init__ self._ip = self._ip_int_from_string(addr_str) File "C:\ProgramData\Anaconda3\lib\ipaddress.py", line 1144, in _ip_int_from_string raise AddressValueError("Expected 4 octets in %r" % ip_str) ipaddress.AddressValueError: Expected 4 octets in 'localhost' ``` But it should just either convert `localhost` to `127.0.0.1` or maybe store `localhost` internally somehow.

2 1

Allow dataclasses's auto-generated __init__ to optionally accept **kwargs and pass them to __post_init_
by thomas.d.mckay＠gmail.com 21 Sep '21

21 Sep '21

Sorry for the double post, if the first one passed... I typed Enter too soon by accident :( TL;DR: Add a `strict` keyword option to the dataclass constructor which, by default (True), would keep the current behavior, but otherwise (False) would generate an __init__ that accepts arbitrary **kwargs and that passes them to an eventual __post_init__. Use case: I'm developing a client for a public API (that I don't control). I'd like this client to be well typed so my users don't have to constantly refer to the documentation for type information (or just to know which attributes exist in an object). So I turned to dataclasses (because they're fast and lead to super clean/clear code). @dataclass class Language: iso_639_1: Optional[str] name: Optional[str] Then my endpoint can look like this def get_language() -> Language: result = requests.get(...) return Language(**result.json) That's fine but it poses a problem if the API, the one I have no control over, decides overnight to add a field to the Language model, say 'english_name'. No change in the API number because to them, that's not a breaking change (I would agree). Yet every user of my client will see "TypeError: __init__() got an unexpected keyword argument 'english_name'" once this change goes live and until I get a chance to update the client code. Other clients return plain dicts or dict wrappers with __get_attr__ functionality (but without annotations so what's the point). Those wouldn't break. I've looked around for solutions and what I found (https://stackoverflow.com/questions/55099243/python3-dataclass-with-kwargsa…) ranged from "you'll have to redefine the __init__, so really you don't want dataclasses" to "define a 'from_kwargs' classmethod that will sort the input dict into two dicts, one for the __init__ and one of extra kwargs that you can do what you want with". Since I'm pretty sure I _do_ want dataclasses, that leaves me with the second solution: the from_kwargs classmethod. I like the idea but I'm not a fan of the execution. First, it means my dataclasses don't work like regular ones, since they need this special factory. Second, it does something that's pretty trivial to do with **kwargs, as we can use **kwargs unpacking to sort parameters instead of requiring at least 2 additional function calls (from_kwargs() and dataclass.fields()), a loop over the dataclass fields and the construction of yet another dict (all of which has a performance cost). My proposal would be to add a `strict=True` default option to the dataclass constructor. the default wouldn't change a thing to the current behavior. But if I declare: @dataclass(strict=False) class Language: iso_639_1: Optional[str] name: Optional[str] Then the auto-generated __init__ would look like this: def __init__(self, iso_639_1, name, **kwargs): ... self.__post_init__(..., **kwargs) # if dataclass has a __post_init__ This would allow us to achieve the from_kwargs solution in a much less verbose way, I think. @dataclass(strict=False) class Language: iso_639_1: Optional[str] name: Optional[str] extra_info: dict = field(init=False) def __post_init__(self, **kwargs) if kwargs: logger.info( f'The API returned more keys than expected for model {self.__class__.__name__}: {kwargs.keys()}. ' 'Please ensure that you have installed the latest version of the client or post an issue @ ...' ) self.extra_info = kwargs I'm not married to the name `strict` for the option, but I think the feature is interesting, if only to make dataclasses *optionally* more flexible. You don't always have control over the attributes of the data you handle, especially when it comes from external APIs. Having dataclasses that don't break when the attributes evolves can be a great safeguard. Outside of my (somewhat specific, I'll admit) use-case, it would also allow dataclasses to be used for types that are inherently flexible. Imagine: @dataclass(strict=False) class SomeTranslatableEntitiy: name: Optional[str] name_translations: dict[str, str] = field(init=False) def __post_init__(self, **kwargs) self.name_translations = { k: kwargs.pop(k) for k, v in kwargs.keys() if k.startswith('name_') # e.g: 'name_en', 'name_fr' } Thanks for reading :)

3 6

Add timeout parameter to Synchronization Primitives in asyncio
by Andres Torres 20 Sep '21

20 Sep '21

It would be very nice if the [Synchronization Primitives](https://docs.python.org/3/library/asyncio-sync.html) had a timeout parameter just like the [analogous classes](https://docs.python.org/3/library/threading.html#condition-objects) do in the threading module. Thank you for your consideration!

4 5

Re: Add timeout parameter to Synchronization Primitives in asyncio
by Blue 20 Sep '21

20 Sep '21

This is already supported with [`asyncio.wait_for()`]( https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for). For example: `await asyncio.wait_for(asyncio.Event().wait(), timeout=60)`

1 0

Otherwise clause in for statement to run when a empty iterable is used
by Andre Delfino 20 Sep '21

20 Sep '21

When working with generators, AFAIK, there's currently no easy way to handle the case of an empty generator (which can be useful if such case is an error). Converting the generator to a, say, list, is not a solution if the generator is intrinsically infinite. I propose to have an "otherwise" clause in the "for" statement that runs only if no items are to be processed; that is, to have the following code: # where get_items() may be an infinite generator for item in get_items(): SUITE1 otherwise: SUITE2 be semantically equivalent to: items_iter = iter(get_items()) try: first_item = next(items_iter) except StopIteration: SUITE2 else: for item in itertools.chain([first_item], items_iter): SUITE1

3 5