Mailman 3 July 2021 - Python-ideas

Why do we have two obvious ways to create a simple data structure? Let's deprecate one.
by pavel＠lexyr.com 29 Jul '21

29 Jul '21

[Migrating the discussion from https://bugs.python.org/issue44768.] PEP 20 says: > There should be one-- and preferably only one --obvious way to do it. There are two ways to create a simple named type to store data: collections.namedtuple and dataclasses.dataclass. I propose deprecating namedtuple. As far as the interface is concerned, the namedtuple is almost completely equivalent to a frozen dataclass - with some iterability syntactic sugar thrown in. I don't think there are use cases for namedtuple that would not be trivial to rewrite with dataclasses. As far as the implementation is concerned, the namedtuple is faster. If efficiency is a concern, why do we make our users decide? We can choose the most efficient one on the library's end. C++ does something similar with bool vectors - the library has a special case for where it would be more optimal to use a different data structure underneath. TL;DR: If dataclass is so good, why keep namedtuple around?

10 21

Make __class__ (and maybe other magic variables) visible to functions that was defined outside a class, but then rebound into a class?
by Yua 28 Jul '21

28 Jul '21

For example, the following code would report an error: class base(): def foo(self) -> None: print('Base!') def placement(self) -> None: super().foo() class child(base): def foo(self) -> None: pass foo = placement child().foo() RuntimeError: super(): __class__ cell not found However, it would be OK if `placement` is defined inside the class: class base(): def foo(self) -> None: print('Base!') class child(base): def placement(self) -> None: super().foo() def foo(self) -> None: pass foo = placement child().foo() which prints: Base! I think it would be natural if those functions that was defined outside a class, but then rebound into a class, can see magic variables like __class__ that are only shared by those functions defined inside a class. Thank you!

3 3

weakref.link: Keep A alive while B is alive (and/or update WeakKeyDictionary)
by Mark Gordon 28 Jul '21

28 Jul '21

Proposal: Have a weakref.link (not at all attached to the naming) primitive that allows one to keep object A alive while object B is alive. This would be functionally similar to adding A as an attribute of B from the GC's perspective without making any assumptions about what attributes are available in B. Here's a really course example of how this might be used: # Associate "extra_data" with "obj" where "extra_data" will lose its reference when "obj" is no longer reachable through strong refs. my_link = weakref.link(obj, extra_data) ... obj, extra_data = my_link() if obj is not None: do_something(obj, extra_data) This would also allow us to fix or create a new version of WeakKeyDictionary where values can have strong references back to their key without preventing their key from being reclaimed. This has been an issue for tensorflow in the past (see https://github.com/tensorflow/tensorflow/issues/42022) and is likely to be a subtle issue in many more libraries. Even the suggested use case in the docs "This can be especially useful with objects that override attribute accesses." is likely to be broken because of this issue. While I'd be weary of changing the semantics of an existing data structure it seems highly unlikely anything depends on this behavior since it makes WeakKeyDictionary degrade to behaving like a normal dictionary for that key. Notes: * Not attached to this specific approach or API * Mostly interesting in updating/creating new WeakKeyDictionary as mentioned

3 3

Picking Up the Ball on Stuck PRs
by Jack DeVries 25 Jul '21

25 Jul '21

Hi! I'm interested in this workflow issue of cpython pull requests that get tons of review and feedback, but, for whatever reason, ultimately lose the attention of the original contributor. I opened a GitHub issue on the dev guide repo where I wrote more about the problem, and proposed some solutions/ideas. I'm hoping that the issue might get more attention if I mentioned it here, too, so I'm wondering if you can take a moment to check it out and continue the conversation on GitHub. https://github.com/python/devguide/issues/731 Thanks!! Jack

1 0

RFC for PEP 663: Improving and Standardizing Enum str(), repr(), and format() behaviors
by Ethan Furman 21 Jul '21

21 Jul '21

PEP: 663 Title: Improving and Standardizing Enum str(), repr(), and format() behaviors Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan(a)stoneleaf.us> Discussions-To: python-dev(a)python.org Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Feb-2013 Python-Version: 3.11 Post-History: 20-Jul-2021 Resolution: Abstract ======== Now that we have a few years experience with Enum usage it is time to update the ``repr()``, ``str()``, and ``format()`` of the various enumerations by their intended purpose. Motivation ========== The addition of ``StrEnum`` with its requirement to have its ``str()`` be its ``value`` is inconsistent with other provided Enum's ``str``. Having the ``str()`` of ``IntEnum`` and ``IntFlag`` not be the value causes bugs and extra work when replacing existing constants. Having the ``str()`` and ``format()`` of an enum member be different can be confusing. The iteration of ``Flag`` members, which directly affects their ``repr()``, is inelegant at best, and buggy at worst. Rationale ========= Enums are becoming more common in the standard library; being able to recognize enum members by their ``repr()``, and having that ``repr()`` be easy to parse, is useful and can save time and effort in understanding and debugging code. However, the enums with mixed-in data types (``IntEnum``, ``IntFlag``, and the new ``StrEnum``) need to be more backwards compatible with the constants they are replacing -- specifically, ``str(replacement_enum_member) == str(original_constant)`` should be true (and the same for ``format()``). IntEnum, IntFlag, and StrEnum should be as close to a drop-in replacement of existing integer and string constants as is possible. Towards that goal, the str() output of each should be its inherent value; e.g. if ``Color`` is an ``IntEnum``:: >>> Color.RED <Color.RED: 1> >>> str(Color.RED) '1' >>> format(Color.RED) '1' Note that format() already produces the correct output in 3.10, only str() needs updating. As much as possible, the ``str()`, ``repr()``, and ``format()`` of enum members should be standardized across the stardard library. The repr() of Flag currently includes aliases, which it should not; fixing that will, of course, already change its ``repr()`` in certain cases. Specification ============= There a three broad categories of enum usage: - standard: Enum or Flag a new enum class is created, and the members are used as ``class.member_name`` - drop-in replacement: IntEnum, IntFlag, StrEnum a new enum class is created which also subclasses ``int`` or ``str`` and uses ``int.__str__`` or ``str.__str__``; the ``repr`` can be changed by using the ``global_enum`` decorator - user-mixed enums and flags the user creates their own integer-, float-, str-, whatever-enums instead of using enum.IntEnum, etc. Some sample enums:: # module: tools.py class Hue(Enum): # or IntEnum LIGHT = -1 NORMAL = 0 DARK = +1 class Color(Flag): # or IntFlag RED = 1 GREEN = 2 BLUE = 4 class Grey(int, Enum): # or (int, Flag) BLACK = 0 WHITE = 1 Using the above enumerations, the following table shows the old and new behavior, while the last shows the final result: +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | 3.10 | | | | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | | | 1 | <Grey.WHITE: 1> | | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | Grey.WHITE | <Grey(1): WHITE> | | Grey.WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | -1 | | <Color(3): RED|GREEN> | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | <Grey.WHITE: 1 | Grey.WHITE | Grey.WHITE | <Grey.WHITE: 1> | Grey.WHITE | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | -1 | | tools.RED|tools.GREEN | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ Which will result in: +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | <Grey.WHITE: 1> | Grey.WHITE | Grey.WHITE | <Grey(1): WHITE> | Grey.WHITE | Grey.WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | <Hue.LIGHT: -1> | -1 | -1 | <Color(3): RED|GREEN> | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | tools.LIGHT | -1 | -1 | tools.RED|tools.GREEN | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ As can be seen, ``repr()`` is primarily affected by whether the members are global, while ``str()`` is affected by being global or by being a drop-in replacement, with the drop-in replacement status having a higher priority. Also, the basic ``repr()`` and ``str()`` have changed for flags as the old style was very clunky. The ``repr()`` for Enum vs Flag are different, primarily because the Enum ``repr()`` does not work well for flags. I like being able to tell whether an enum member is a Flag or an Enum based on the ``repr()`` alone, but am open to arguments for changing Enum's ``repr()`` to match Flag's. Backwards Compatibility ======================= My understanding is that ``str()`` and ``repr()`` output has much lower backwards compatibility requirements. Even so, I expect the majority of breakage to be in doc and unit tests. I'm less clear on the policy for ``format()``. Note that by changing the ``str()`` of the drop-in category, we will actually prevent future breakage when ``IntEnum``, et al, are used to replace existing constants. Mitigation ========== Normal usage of enum members will not change: ``re.ASCII`` can still be used as ``re.ASCII`` and will still compare equal to ``256``. If one wants their own enums in their own code to remain the same they will need to write their own base Enum class and then write the appropriate ``repr``, ``str()``, and ``format()`` methods (or copy them from the 3.10 enum module). Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

1 0

Nested Dictionary Pointers and Logic - "flatten / nested iteration" Proposal
by Sven Voigt 21 Jul '21

21 Jul '21

Hello Python-ideas, I am working with lots of JSON objects in my work and need to obtain JSON pointers to particular components of a JSON object. These pointers are defined as part of the JSON-LD specifications as relative IRIs and as part of the JSON-Schema specifications. However, I am not aware of these pointers being part of the JSON specification. The pointers are really straightforward and follow the same syntax as other IRIs (URLs) or file paths. For example, the following flatten function illustrates the mapping of nested dictionaries and arrays to a single flat dictionary, where each nested key is mapped to a unique path. d = {"a": 1, "b": {"ba": 2, "bb": [{"bba": 3, "bbb": None}]}} flatten(d) >> {'a': 1, 'b/ba': 2, 'b/bb/0/bba': 3, 'b/bb/0/bbb': None} Not only does this conversion help in generating JSON pointers, but it helps with logic for nested data structures. Specifically, assume there is a dictionary, which includes nested dictionaries, lists, and tuples, and any of these elements are contained within another dictionary or that other dictionaries elements. Then it is not sufficient to simply compare the items of the two dictionaries to check whether the first is a subset of the other. However, the flat structures can be compared directly. 1. A nested dictionary is a subset of another nested dictionary. The flatten function ensures the nested dictionaries are not checked for equivalence but subset of. # Current {"a": 1, "c": {"d": 4}}.items() <= {"a": 1, "b": 2, "c": {"d": 4, "e": 5}}.items() >> False # Proposed flatten({"a": 1, "c": {"d": 4}}).items() <= flatten({"a": 1, "b": 2, "c": {"d": 4, "e": 5}}).items() >> True 2. A nested list or tuple is a subset of a dictionary. # Current {"a": 1, "c": [3]}.items() <= {"a": 1, "b": 2, "c": [3,3]}.items() >> False # Proposed flatten({"a": 1, "c": [3]}).items() <= flatten({"a": 1, "b": 2, "c": [3,3]}).items() >> True Note that these examples only have one level of nesting, but the flatten function must handle any level of nesting. For the current version of the flatten function, I essentially borrowed the JSON "dumps" source code. However, the dumps code doesn't utilize any other functions and I believe there may be a missing "nestedIter" function in python. This function could be used for JSON dumps or flatten or anytime someone wants to visit all nested elements of a dictionary and perform some operation. Therefore, I think this function could be included in the JSON library if python-ideas thinks that the pointers are very specific to JSON or the itertools library, where the nested iteration function could be generally used for all nested data types. Let me know what you think of this proposal and I am looking forward to your responses. Best, Sven

5 10

binascii generalization
by Ronald Hoogenboom 20 Jul '21

20 Jul '21

I see that the binascii module has several {a2b,b2a}_xyz functions for several flavors of ascii <-> binary conversions. But it misses a general base function set parameterized with a radix. It would be nice to add a generic set of {a2b|b2a}(s, radix=r, case={'u','l'}), where radix is a power of 2 < 36 (2, 4, 8, 16 or 32) and case is only significant for b2a, to cover all trivial ascii <-> binary conversions using an efficient native implementation. Most {a2b,b2a}_xyz can just call the generic {a2b|b2a} with the appropriate radix. 'radix' could also be called 'base' for consistency with the int() functions. I was actually surprised that it wasn't the case already...

1 0

builtins for running context managers
by Thomas Grainger 17 Jul '21

17 Jul '21

currently lots of code manages contexts incorrectly by doing: ``` @dataclasses.dataclass class WrapCmgr: _cmgr: ContextManager[T] def __enter__(self) -> Wrapped[T]: return wrap(_cmgr.__enter__()) def __exit__(self, \, t: Type[BaseException] | None, v: BaseException, tb: types.TracebackType) -> bool: return _cmgr.__exit__(t, v, tb) ``` since https://bugs.python.org/issue44471 using `with WrapCmgr(cmgr()):` incorrectly raises an AttributeError rather than a TypeError and https://www.python.org/dev/peps/pep-0343/ still includes this bug, raising an AttributeError ``` mgr = (EXPR) exit = type(mgr).__exit__ # Not calling it yet value = type(mgr).__enter__(mgr) ``` contextlib.ExitStack also had this bug https://bugs.python.org/issue12022 given that it's hard to implement this correctly I think there should be a builtins.enter and builtins.aenter for use like this: ``` @dataclasses.dataclass class WrapCmgr: _cmgr: ContextManager[T] def __enter__(self) -> Wrapped[T]: exit, value = enter(self._cmgr) self.__exit = exit return wrap(value) def __exit__(self, \, t: Type[BaseException] | None, v: BaseException, tb: types.TracebackType) -> bool: return self.__exit(t, v, tb) ```

5 12

Looking for a PEP sponsor
by Nick Humrich 15 Jul '21

15 Jul '21

Hello Python Devs, In accordance to PEP 0, I am looking for a Python-core-developer to sponsor my new PEP. It is essentially a revisit of PEP 501. https:// github.com/nhumrich/peps/blob/master/pep-9999.rst <https://t.co/cCrqfWztAW?amp=1> Please let me know if you have questions, or would be interested in sponsoring this PEP. Nick Humrich

3 2

Advanced RGB Coloring
by z3ntl3discord＠gmail.com 14 Jul '21

14 Jul '21

Hey dear community, I want to use advanced RGB Coloring in python for teminals. There are already some modules but not advancrd enough. I hope this feature will be added in the next python versions! ❤ I believe in this strong community, this will powerup our ascii designs in terminals most users made!

3 2