Mailman 3 September 2021 - Python-ideas

Re: Different exceptions for assert
by Steven D'Aprano 12 Sep '21

12 Sep '21

On Fri, Sep 10, 2021 at 06:48:42PM -0400, Juancarlo Añez wrote: > I agree that assertions can be and are abused. > > And I'm not too interested in DBC, nor in preconditions, nor in contracts. In your first post you said: [quote] This is in the context of "Design by contrast" (DBC) as a useful companion to "Test-driven development" and other forms of external tests. [/quote] I presume the "contrast" was a typo for contract. > I'd like to make it easy to assert invariants and postconditions Invariants and postconditions **are** contracts. Both are part of DBC. It seems very odd that you are taking only two thirds of DBC. Why do you think that preconditions are not important? > over critical pieces of code to guard against failures caused by: > > - bugs in the design or the implementation > - third party software violating its contracts > - the environment You just said you're not interested in contracts. Now you say you want to enforce third-party software's contracts. The first item you give (bugs in design or implementation) is part of DBC. The second is not something you can enforce. You don't know what, if any, contracts a third-party library may be enforcing internally. The only thing you can even *attempt* to enforce is the third-party external API, but you don't need anything more than a bare assert for that. You can use assert as it is to validate your arguments that you pass to libraries: assert isinstance(count, int) and count > 0 result = some_library.func(count) so no change needed there. To verify that the third-party library is returning what it promises to return can also be done with assert: assert result is not None and hasattr(result, 'spam') > Why target *assert*? Because invariants are clearer when the expected > condition is what's asserted. That's arguable. If all you want is to be able to flip the sense of a test without using `not`, then it sounds like you need an "unless" keyword: unless condition: # block raise MyError(msg) That avoids abusing assert for things that aren't assertions. But adding a new keyword just so you don't have to write `not` seems like a high cost for a low benefit. Although now that we have soft keywords, the cost is somewhat less. > Why specify an ExceptionType? Although in theory an assertion should never > fail, reality is that they will fail, If your assertions are regularly failing in production, then either your application is full of bugs and you put it into production too soon, or you are misusing assert by using it for things which aren't assertions. In any case, you don't need to raise different exception types in order to assess the failure. You need to know *where* the failure occurred. > and it's useful for outer layers of a > software system to be able to assess the severity of a failure to know > which actions are appropriate. > > - HALT > - preserve major state and HALT > - rollback and halt > - retry > - ... The first three are what I would call "exit gracefully" (if possible, but if an error is so severe you can't exit gracefully, then all you can do is exit). If an error can be resolved by retrying, then it is not an assertion that was violated. You cannot *assert* that (e.g.) a web server will respond with the expected amount of data, or that a user will input exactly 'Y' or 'N' at the prompt. -- Steve

1 0

PEP 663: Improving and Standardizing Enum str(), repr(), and format() behaviors
by Ethan Furman 11 Sep '21

11 Sep '21

PEP 663 is presented below for your viewing pleasure. Comments, questions, and concerns are welcome. -------------------------------------------------------------------------------- PEP: 663 Title: Improving and Standardizing Enum str(), repr(), and format() behaviors Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan(a)stoneleaf.us> Discussions-To: python-dev(a)python.org Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Feb-2013 Python-Version: 3.11 Post-History: 20-Jul-2021 10-Sep-2021 Resolution: Abstract ======== Now that we have a few years experience with Enum usage it is time to update the ``repr()``, ``str()``, and ``format()`` of the various enumerations by their intended purpose. Motivation ========== The addition of ``StrEnum`` with its requirement to have its ``str()`` be its ``value`` is inconsistent with other provided Enum's ``str``. Having the ``str()`` of ``IntEnum`` and ``IntFlag`` not be the value causes bugs and extra work when replacing existing constants. Having the ``str()`` and ``format()`` of an enum member be different can be confusing. The iteration of ``Flag`` members, which directly affects their ``repr()``, is inelegant at best, and buggy at worst. Rationale ========= Enums are becoming more common in the standard library; being able to recognize enum members by their ``repr()``, and having that ``repr()`` be easy to parse, is useful and can save time and effort in understanding and debugging code. However, the enums with mixed-in data types (``IntEnum``, ``IntFlag``, and the new ``StrEnum``) need to be more backwards compatible with the constants they are replacing -- specifically, ``str(replacement_enum_member) == str(original_constant)`` should be true (and the same for ``format()``). IntEnum, IntFlag, and StrEnum should be as close to a drop-in replacement of existing integer and string constants as is possible. Towards that goal, the str() output of each should be its inherent value; e.g. if ``Color`` is an ``IntEnum``:: >>> Color.RED <Color.RED: 1> >>> str(Color.RED) '1' >>> format(Color.RED) '1' Note that format() already produces the correct output in 3.10, only str() needs updating. As much as possible, the ``str()`, ``repr()``, and ``format()`` of enum members should be standardized across the stardard library. The repr() of Flag currently includes aliases, which it should not; fixing that will, of course, already change its ``repr()`` in certain cases. Specification ============= There a three broad categories of enum usage: - standard: Enum or Flag a new enum class is created, and the members are used as ``class.member_name`` - drop-in replacement: IntEnum, IntFlag, StrEnum a new enum class is created which also subclasses ``int`` or ``str`` and uses ``int.__str__`` or ``str.__str__``; the ``repr`` can be changed by using the ``global_enum`` decorator - user-mixed enums and flags the user creates their own integer-, float-, str-, whatever-enums instead of using enum.IntEnum, etc. Some sample enums:: # module: tools.py class Hue(Enum): # or IntEnum LIGHT = -1 NORMAL = 0 DARK = +1 class Color(Flag): # or IntFlag RED = 1 GREEN = 2 BLUE = 4 class Grey(int, Enum): # or (int, Flag) BLACK = 0 WHITE = 1 Using the above enumerations, the following table shows the old and new behavior, while the last shows the final result: +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | 3.10 | | | | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | | | 1 | <Grey.WHITE: 1> | | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | Grey.WHITE | <Grey(1): WHITE> | | Grey.WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | -1 | | <Color(3): RED|GREEN> | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | <Grey.WHITE: 1 | Grey.WHITE | Grey.WHITE | <Grey.WHITE: 1> | Grey.WHITE | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | -1 | | tools.RED|tools.GREEN | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ Which will result in: +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | <Grey.WHITE: 1> | Grey.WHITE | Grey.WHITE | <Grey(1): WHITE> | Grey.WHITE | Grey.WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | <Hue.LIGHT: -1> | -1 | -1 | <Color(3): RED|GREEN> | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | tools.LIGHT | -1 | -1 | tools.RED|tools.GREEN | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ As can be seen, ``repr()`` is primarily affected by whether the members are global, while ``str()`` is affected by being global or by being a drop-in replacement, with the drop-in replacement status having a higher priority. Also, the basic ``repr()`` and ``str()`` have changed for flags as the old style was very clunky. The ``repr()`` for Enum vs Flag are different, primarily because the Enum ``repr()`` does not work well for flags. I like being able to tell whether an enum member is a Flag or an Enum based on the ``repr()`` alone, but am open to arguments for changing Enum's ``repr()`` to match Flag's. Backwards Compatibility ======================= Backwards compatibility of stringified objects is not guarenteed across major Python versions, and there will be backwards compatibility breaks where software uses the ``repr()``, ``str()``, and ``format()`` output of enums in tests, documentation, data structures, and/or code generation. Normal usage of enum members will not change: ``re.ASCII`` can still be used as ``re.ASCII`` and will still compare equal to ``256``. If the previous output needs to be maintained, for example to ensure compatibily between different Python versions, software projects will need to create their own enum base class with the appropriate methods overridden. Note that by changing the ``str()`` of the drop-in category, we will actually prevent future breakage when ``IntEnum``, et al, are used to replace existing constants. Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

1 0

Iterate through members of enum.Flag
by matthew＠matthewstocks.co.uk 10 Sep '21

10 Sep '21

My proposal is that iterating through a member of a Flag enum will return all the constituent members. Demonstration Setup: class FlagEnum(enum.Flag): A = enum.auto() B = enum.auto() C = enum.auto() FlagCombo = FlagEnum.A | FlagEnum.B My proposed change would cause the following to occur: >>> print(list(FlagCombo)) [FlagEnum.B, FlagEnum.A] >>> print(list(FlagEnum.A)) [FlagEnum.A] Compared to the current implementation: >>> print(list(FlagCombo)) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'FlagEnum' object is not iterable What do you all think?

2 1

Add check_methods function to standard library
by Finn Mason 09 Sep '21

09 Sep '21

Hello, In _collections_abc.py is a private function titled `_check_methods`. It takes a class and a number of method names (as strings), checks if the class has all of the methods, and returns NotImplemented if any are missing. The code is below: ``` def _check_methods(C, *methods): mro = C.__mro__ for method in methods: for B in mro: if method in B.__dict__: if B.__dict__[method] is None: return NotImplemented break else: return NotImplemented return True ``` This is an incredibly convenient function (referred to as check_methods here on out) for creating abstract base classes, and is much simpler than using `hasattr` for each method you want to check. For example: ``` >>> from abc import ABCMeta >>> # Without check_methods >>> class A(metaclass=ABCMeta): ... @classmethod ... def __subclasshook__(cls, subclass): ... return (hasattr(subclass, 'foo') and ... callable(subclass.foo) and ... hasattr(subclass, 'bar') and ... callable(subclass.bar) or ... NotImplemented) ... >>> # With check_methods >>> class B(metaclass=ABCMeta): ... @classmethod ... def __subclasshook(cls, subclass): ... return check_methods(subclass, 'foo', 'bar') ... >>> ``` This would be a great function to add to the standard lib, perhaps in the `abc` module. One problem with `check_methods` as defined in _collections_abc.py is that it doesn't check if the name is callable. Also, type hints and more readable variables may be desirable. The final code, if implemented, may look something like this: ``` # In imports section: from typing import Literal def check_methods(Class: type, *methods: str) -> Literal[True, NotImplemented]: """Check if class `Class` has methods `methods`.""" mro = Class.__mro__ for method in methods: for Base in mro: if (attr := getattr(Base, method, None)) is not None: if not callable(attr): return NotImplemented break else: return NotImplemented return True ``` Again, this would be a great function to add to the `abc` module or a similar one. I've proposed this in issue 44941. Another possible implementation may be similar to a dataclass that implements __subclasshook__ for you with the desired method checks. --Finn

1 2

Additional Unpacking Generalizations for assignment
by 笹原康央 09 Sep '21

09 Sep '21

Hello developers. I like star expression. It would be even better if we could extend the idea of PEP448 to provide the following syntax at the time of assignment. ``` python a, b, d, **_ = **{"a":0, " b":1, "c":2}, d=3 ``` Ideally, it would be nice to be able to provide the same syntax when calling the function. I think this applies to the zen of python called "There should be one-- and preferably only one --obvious way to do it.". ``` python a, *args, b, **kwargs = 1, 2, *[3,4], **{"b", 4}, c=4 ( a: str, *args, b: list = [], **kwargs ) = 1, 2, *[3,4], **{"b", 4}, c=4 func = lambda( a: str, *args, b: list = [], **kwargs ): print(a, b) ``` But it would be difficult because it would conflict with various syntaxes(etc. PEP 3132). We may have to declare it explicitly to avoid confusion. For example, prepare a keyword for unpack the positional arguments and keyword arguments at the same time. ``` a, *args, b, **kwargs = ***(1, 2, *[3,4], **{"b", 4}, c=4) ``` You can now explicitly recognize that you are using the behavior of the new assignment. I hope this is an idea to develop python. Thanks.

2 3

__name__ for partial functions
by Henry Harutyunyan 07 Sep '21

07 Sep '21

functools introduces partial function. As I can see from the implementation, partial functions are instance of `partial` class, and that __call__ is used to "emulate" a function call. On the other hand I guess people usually treat those as ordinary functions. The thing here is that for functions people expect to see the __name__ attribute defined and it can be useful in many cases. Why partial does not have a __name__ property and why is it not a good idea to add one?

2 1

open functions in dbm submodule need to support path-like object
by Evan Greenup 07 Sep '21

07 Sep '21

Currently, in Python 3.9, `dbm.open()`, `dbm.gnu.open()` and `dbm.ndbm.open()` doesn't support path-like object, class defined in `pathlib`. It would be nice to add support with it. Sent with ProtonMail Secure Email.

3 2

Context manager for csv module 'reader' and 'DictReader'?
by C. Titus Brown 07 Sep '21

07 Sep '21

Hi all, the product of Sunday morning idle curiosity... I’ve been using the csv module a lot, and I’m wondering if there would be value in adding a standard mechanism for opening a CSV file (correctly) using a context manager? So, instead of with open(filename, newline=“”) as fp: r = csv.DictReader(fp) for row in r: … support something like with csv.DictReader.open(filename) as r: for row in r: … ? And something similar for ‘csv.reader’? I’m not wedded to the details here. The two main reasons I think this might be a positive addition are - * you wouldn’t have to know or remember the right way to open a CSV file (newline=“”). * it elides very common code. but perhaps there are things I’m missing here? As a side note, I think ‘csv.reader’ could usefully be renamed to something else (maybe just Reader?), since it’s kind of out of sync with the CamelCase used in ‘DictReader’. But maybe that’s just an attempt at foolish consistency :). best, —titus

8 12

Improve traceback for common `with` statement mistake
by Finn Mason 06 Sep '21

06 Sep '21

Hello all, In Python 3.10 and 3.11, exception tracebacks are being greatly improved. I noticed that there's nothing related to a fairly common (in my personal experience) cryptic traceback relating to the `with` statement: >>> with ContextManager as ctx: ... # do something with `ctx` ... Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: __enter__ This occurs when one forgets to use a instance of a context manager class and uses the class itself. It's obviously not a very helpful traceback. ("Is it not a context manager?" "Is it the wrong class?") Something like the following would be better. >>> with ContextManager as ctx: ... # do something with `ctx` ... Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: <type Context manager> is not a context manager. Did you mean "with ContextManager()..."? The actual traceback message should probably be more specific than "<type Context manager> is not a context manager". Thoughts? -- Finn

3 4

Allow starred expressions to be used as subscripts for nested lookups for getitem and setitem
by Kevin Mills 05 Sep '21

05 Sep '21

Given code like this: ``` d = {1: {2: {3: 4}}} print(d[1][2][3]) d[1][2][3] = None print(d) ``` It should be possible to rewrite it using a starred expression, like this: ``` d = {1: {2: {3: 4}}} keys= 1,2,3 print(d[*keys]) d[*keys] = None print(d) ``` Hopefully it's clear from that example what I'm suggesting.

7 14