I think it will be an act of kindness to
deprecate Py_TRASHCAN_SAFE_BEGIN/END in 3.10 and tell people to use
TL;DR: There was a change in 3.8 that introduced the latter while leaving
the former for backwards compatibility, but also inadvertently breaking
them. This is not an easy bug to deal with in the wild, we found it because
we have a unit test in our codebase referencing https://
bugs.python.org/issue16602. A deprecation note pointing to the new macros
would have made it easier.
Is there any reason not to deprecate the old macros?
We've got to the stage now with PEP 646 that we're feeling pretty happy
with it. So far though we've mainly been workshopping it in typing-sig, so
as PEP 1 requires we're asking for some feedback here too before submitting
it to the steering council.
If you have time over the next couple of weeks, please take a look at the
current draft and let us know your thoughts:
https://www.python.org/dev/peps/pep-0646/ (Note that the final couple of
sections are out of date; https://github.com/python/peps/pull/1880
clarifies which grammar changes would be required, now that PEP 637 has
been rejected. We also have a second PR in progress at
https://github.com/python/peps/pull/1881 clarifying some of the motivation.)
Matthew and Pradeep
As you might know, PEP 581 (Using GitHub Issues for CPython) has been
approved. I've been working with Ewa, Ee, the Working Group, the
Steering Council, and the GitHub folks to make this happen, and the SC
encouraged me to give you all a quick update.
This effort is being tracked at
<https://github.com/psf/gh-migration/projects/1>: this board reflects
the current status of the project. The PEPs (including PEP 588 --
GitHub Issues Migration Plan) haven't been updated yet and might
contain outdated information, so please refer to the psf/gh-migration
repo for the latest updates.
During the next phase I will work with the WG to sort out all the
major issues that we might encounter, and then I will once again reach
out to you to gather feedback from the wider audience that follows
these mailing lists.
I have spoken with Pablo (3.10 RM), and he agrees that a change to Enum str() in 3.10 and another in 3.11 is less than
ideal, so this new thread is to collect comments about Enum and it's str() and repr() and whether the changes take
effect in 3.10, 3.11, or both.
TL;DR -- sorry, there isn't one.
As Enum and IntEnum have started proliferating in the stdlib, some of those enumerations have changed the str() and
repr() of their members -- for example, in 3.8 re.RegexFlag had its repr() and str() changed to be module.name
(re.ASCII) for both instead of <enum_class.name: value> (<RegexFlag.ASCII: 256>) for the repr(); that change made sense
because all the members are exported to re's global name space, and they are accessed as `re.ASCII` in existing code.
While a good change (in my opinion and with which I had nothing to do), we now have stdlib enumerations with differing
str() and repr()s, which is a minor ding against recognizability and usability.
Current 3.10 Changes
In an effort to standardize the stdlib enums, a new decorator was added: global_enum(). It can be called manually (for
example in re.RegexFlag), or automatically by Enum._convert_ (for example in ssl.VerifyFlags).
That changed was visually appealing, and I had users wanting that type of output for Enum in general, so after asking
for feedback on python-dev and python-ideas (and receiving little) I changed the basic str() and repr() of Enum to:
- str(): NAME
- repr(): enum.NAME
While working on some other bugs/changes in Enum.__format__ and serialization in general, I have come to believe that
IntEnum (and IntFlag) should be as near-perfect a drop-in replacement as possible for when they are used to replace
existing integer constants (which I believe is their most important use-case).
Reverting 3.10 Changes
I have been increasingly unhappy with the general `Enum.__repr__` change, so while the stdlib global repr() change is
staying, Enum, IntEnum, etc., is going back to <enum_class.NAME: value>.
Proposed 3.10 Change (for 3.10.0 beta 4)
In order to improve the drop-in replacement use-case for IntEnum and IntFlag, I would like to change the str() for those
two to be simply the value of the member, as if it was still a plain integer and not part of an enumeration. At that
point the only obvious difference would be the repr(), which I think is the most important and useful change as that is
what (should) show up in log files, the REPL, etc.
This would also make IntEnum and IntFlag consistent with StrEnum as, for other reasons, StrEnum member str()s are the
values of the members rather than the names of the members. Note also that this change would not affect user created
enumerations that mixed in int on their own.
With all the changes currently happening to the str()s and repr()s of the various enums, what are the good reasons to
not make this final change now, and instead have one more change in 3.11?
Summary of Enum str()s and repr()s by category
| | repr() | str() | format()
| stdlib global | re.ASCII | re.ASCII | 256
| Enum | <Color.RED: 1> | Color.RED | RED
| IntEnum | <Color.RED: 1> | 1 | 1
| int, Enum | <Color.RED: 1> | Color.RED | 1 (will be RED in 3.12 due to back-compatibility
| | | constraints which should affect very few people as
| | | as they are probably using IntEnum instead of mixing
| | | in int themselves -- comments about this likelihood also
| | | appreciated)
I just stumbled upon the following issue and subsequent pull request. It is a very small bugfix. There is currently a bug in Python and this pull request fixes it. It's not a new feature or an enhancement, it is a bugfix! Yet, it doesn't get reviewed, nor merged. And this has been going on since March 2017. Why not just merge this? It's not like it's huge or obstructing or obfuscating the current code base? There's always time to write an improvement or a complete rewrite of this module feature later for an upcoming minor release. But if there is currently a bug in Python and the bugfix is available - why doesn't it get merged?
If this doesn't get fixed, doesn't that mean the Python review process is flawed? Sure, Python is an open source project and many people just work in their free time. But this doesn't really apply here, does it? The bugfix is available. Only a review is required. All this is happening while new features get added to Python with every new minor version. While the bug is allowed to live there. Please help me understand how this can happen.
I love Python. No hard feelings. But this is really bugging me and I can't help but feel disappointed.
Apologies for my tardiness in doing this, but no one explicitly said it
was too late to critique PEP 657...
Critique of PEP 657 (Include Fine Grained Error Locations in Tracebacks)
First of all I want to say that I support the goal of improving error
messages. IMO the PEP should be "accepted in principle". I think all of
the issues below can be fixed while still supporting the general aims of
The change from points to ranges as locations
Because Python is a procedural language, there is an expectation that
code executes in a certain order. PEP 626 (Precise line numbers for
debugging and other tools) seeks to guarantee that that expectation is met.
PEP 657 describes how locations for exceptions are to be handled, but is
vague on the treatment of locations for tracing, profiling and debugging.
PEP 657 proposes that locations for exceptions be treated as ranges,
whereas tracing, profiling and debugging currently treat locations as
Either this will end in contradictions and confusion should those
locations disagree, or the locations for tracing, profiling and
debugging must change.
Using the start of a range as the point location for tracing may be
misleading when the operation that causes an exception is on a different
line within that range.
Consider this example:
This might seem like a contrived case, but it is based on a real bug
1. def load_method():
2. return (
Currently the call is traced on line 4.
PEP 657 would change the location of the call from line 4 to the range
3-6, which would mean that the line of call is no longer traced
separately (or traced several times). PEP 657 makes no mention of this
The PEP claims that these changes are improvements. Maybe they are, but
they are quite impactful changes which the PEP glosses over. The impact
on tools like coverage.py and debuggers should be made clearer. For
example, how would one set a breakpoint on line 4 above?
There are other languages (e.g. jinja templates) that compile to Python
AST and bytecode. These *might* produce locations that overlap, but are
not nested. The behavior of tracing and debuggers needs to be described
for those locations.
PEP 657 claims it is fully backwards compatible, but it cannot be both
backwards compatible and consistent.
There are fundamental differences between using ranges and points as
Impact on startup time
The PEP 657 suggests the impact on startup would be negligible. That is
not quite true. The impact on startup is probably acceptable, but a
proper analysis needs to be made.
The increase in size of pyc files ~20% puts an upper bound on the
increase of startup time, but I would expect it to be much less than
that as loading files from disk is only a fraction of startup.
Currently, startup is dominated by inefficiencies in interpreter
creation, unmarshalling and module loading.
We plan to reduce these a lot for 3.11, so that the impact of PEP 657 on
startup will be larger (as a ratio) than experiments with 3.10 suggest.
The C API adds three new functions, one each for the end line, start
column and end column.
This is either slow, as any compressed table needs to be parsed four
times, or space inefficient using an uncompressed table.
Allowing opt-out prevents consistent compression of location data,
resulting in larger pyc files for those that do not opt-out.
The exact semantics, in terms of error formatting, tracing, etc is not
described should the user opt-out.
Overall, there is nothing that blocks acceptance of the PEP in
principle, but there are quite a few issues that need resolving.
1. Clarify, in detail, the impact on line-based tools like profilers,
coverage.py and debuggers. This should include help on how to use the
new APIs and where using the old APIs might result in behavioral changes.
2. Change the C API to a single function:
int PyCode_Addr2Location(PyCodeObject *co, int addr, int *startline, int
*startcolumn, int *endline, int *endcolumn)
3. Drop the opt-out option.
If the extra information is optional, then the compression scheme must
allow for that; making the code more complex and potentially less
efficient. Does opting out use the start of the range, or the old line,
as the location?
4. Drop the limitation on column offsets.
The data needs to be compressed anyway, so allowing arbitrary column
offsets is effectively free.
6. Store all location information in a single table (this applies more
to the implementation than the PEP)
Using four separate objects to hold the location info adds a lot of
overhead for most functions.
I was expected the announcement of a BDFL delegate for PEP 657, as the
author is a steering council member.
It seems that a PEP submitted by a SC member has been accepted by the SC
with seemingly no external review.
PEP 657 is likely to cause significant worsening of start up time and
interact poorly with debugging and profiling.
I’m happy to announce that PEP 657 (Include Fine Grained Error Locations in Tracebacks) has been unanimously accepted for Python 3.11 by the Python Steering Council.
Congratulations Pablo, Batuhan, and Ammar!
-Barry (on behalf of the Steering Council)