Python-Dev
Threads by month
- ----- 2025 -----
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2003 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2002 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2001 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2000 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1999 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
November 2021
- 74 participants
- 45 discussions
Tcl/Tk updates
With the recent release of macOS 12 Monterey, we noticed that tkinter file dialogs are failing to show up on this new operating system, including in our built-in IDLE. Thanks to rapid help from the Tk team, and Marc Culler in particular, we were able to fix the issue by bundling Python 3.9.8 and Python 3.11.0a2 with a fixed Tcl/Tk version. In 3.9.8 it’s a patched 8.6.11 release while 3.11.0a2 is rocking the bleeding-edge 8.6.12rc1.
Since the issue also affected our latest …
[View More]stable version, 3.10.0, an updated macOS installer for this version was issued. You can recognize it by the post2 version appendix: python-3.10.0post2-macos11.pkg <https://www.python.org/ftp/python/3.10.0/python-3.10.0post2-macos11.pkg>. We didn’t have to bump the version number of Python itself since there are no Python source differences between this package and the original 3.10.0. The only difference is the bundled patched Tcl/Tk library.
Initially the original 3.10.0 installer was removed from the website after all URLs got updated to point to the patched version but it turned out this breaks some workflows <https://discuss.python.org/t/disappearing-macos-packages-on-python-org/11737> so the patched installer is now also available under the original filename.
<https://discuss.python.org/t/python-3-9-8-and-3-11-0a2-are-now-available/11…>Python 3.9.8
Get it here: https://www.python.org/downloads/release/python-398/ <https://www.python.org/downloads/release/python-398/>
Python 3.9.8 is the seventh maintenance release of the legacy 3.9 series. Python 3.10 is now the latest feature release series of Python 3. Get the latest release of 3.10.x here <https://python.org/downloads/>.
There’s been 202 commits since 3.9.7 which shows that there’s still considerable interest in improving Python 3.9. To compare, at the same stage of the release cycle Python 3.8 received over 25% fewer commits. See the changelog <https://docs.python.org/release/3.9.8/whatsnew/changelog.html> for details on what changed.
On macOS, the default installer is now the new universal2 variant. It’s compatible with Mac OS X 10.9 and newer, including macOS 11 Big Sur and macOS 12 Monterey. You may need to upgrade third-party components, like pip, to the newest versions. You may experience differences in behavior in IDLE and other Tk-based applications due to using the newest version of Tk. As always, if you encounter problems when using this installer variant, please check https://bugs.python.org <https://bugs.python.org/> for existing reports and for opening new issues.
The next Python 3.9 maintenance release will be 3.9.9, currently scheduled for 2022-01-03.
<https://discuss.python.org/t/python-3-9-8-and-3-11-0a2-are-now-available/11…>Python 3.11.0a2
Get it here: https://www.python.org/downloads/release/python-3110a2/ <https://www.python.org/downloads/release/python-3110a2/>
Python 3.11 is still in development. This release, 3.11.0a2 is the second of seven planned alpha releases.
Alpha releases are intended to make it easier to test the current state of new features and bug fixes and to test the release process.
During the alpha phase, features may be added up until the start of the beta phase (2022-05-06) and, if necessary, may be modified or deleted up until the release candidate phase (2022-08-01). Please keep in mind that this is a preview release and its use is not recommended for production environments.
Many new features for Python 3.11 are still being planned and written. Among the new major new features and changes so far:
PEP 657 <https://www.python.org/dev/peps/pep-0657/> – Include Fine-Grained Error Locations in Tracebacks
The Faster CPython Project 1 <https://github.com/faster-cpython> is already yielding some exciting results: this version of CPython 3.11 is ~12% faster on the geometric mean of the PyPerformance benchmarks <>, compared to 3.10.0.
(Hey, fellow core developer, if a feature you find important is missing from this list, let Pablo know <mailto:pablogsal@python.org>.)
The next pre-release of Python 3.11 will be 3.11.0a3, currently scheduled for Monday, 2021-12-06.
<https://discuss.python.org/t/python-3-9-8-and-3-11-0a2-are-now-available/11…>We hope you enjoy the new releases
Your friendly release team,
Ned Deily @nad <https://discuss.python.org/u/nad>
Pablo Galindo Salgado @pablogsal <https://discuss.python.org/u/pablogsal>
Steve Dower @steve.dower <https://discuss.python.org/u/steve.dower>
Łukasz Langa @ambv <https://discuss.python.org/u/ambv>
[View Less]
1
0
ACTIVITY SUMMARY (2021-10-29 - 2021-11-05)
Python tracker at https://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open 7259 (+10)
closed 50203 (+46)
total 57462 (+56)
Open issues with patches: 2871
Issues opened (37)
==================
#45641: Error In opening a file through tkinter on macOS Monterey
https://bugs.python.org/issue45641 reopened by devesh.dk373
#45675: …
[View More]pkgutil.get_data() doesn't add subpackages to parent packages
https://bugs.python.org/issue45675 opened by godlygeek
#45676: Enum: improve generics support
https://bugs.python.org/issue45676 opened by ethan.furman
#45677: [doc] improve sqlite3 docs
https://bugs.python.org/issue45677 opened by erlendaasland
#45680: Documentation on `GenericAlias` objects and `__class_getitem__
https://bugs.python.org/issue45680 opened by AlexWaygood
#45681: tkinter breaks on high resolution screen after ctypes SetProce
https://bugs.python.org/issue45681 opened by GabeMillikan
#45684: `functools.singledispatchmethod` does not define `__class_geti
https://bugs.python.org/issue45684 opened by AlexWaygood
#45686: ElementTree.Element.extend: bad error message when error occur
https://bugs.python.org/issue45686 opened by goodmami
#45687: Infinite recursion in Pickler.persistent_id
https://bugs.python.org/issue45687 opened by embe-navalgo
#45689: Custom Name for ThreadPoolExecutor
https://bugs.python.org/issue45689 opened by RahulARanger
#45690: Argparse exclusive group inside required exclusive group displ
https://bugs.python.org/issue45690 opened by acooke
#45691: Partial moving of core objects to interpreter state is incorre
https://bugs.python.org/issue45691 opened by Mark.Shannon
#45692: IDLE: define word/id chars in one place.
https://bugs.python.org/issue45692 opened by terry.reedy
#45693: `loop.create_server` with port=0 uses different ports for ipv4
https://bugs.python.org/issue45693 opened by jcristharif
#45694: Limit the number of chained exceptions included in formatted t
https://bugs.python.org/issue45694 opened by iritkatriel
#45695: Out-of-tree builds are not tested.
https://bugs.python.org/issue45695 opened by eric.snow
#45696: "Deep-freeze": skip the marshal step by generating C code
https://bugs.python.org/issue45696 opened by gvanrossum
#45699: AttributeError: 'list' object has no attribute 'find'
https://bugs.python.org/issue45699 opened by krisp1506
#45701: Add tuple tests to `functools.lru_cache`
https://bugs.python.org/issue45701 opened by sobolevn
#45703: importlib.invalidate_caches() does not invalidate _NamespacePa
https://bugs.python.org/issue45703 opened by hroncok
#45704: string.Formatter.parse does not handle auto-numbered positiona
https://bugs.python.org/issue45704 opened by SDesch
#45706: Add IMAP4.login_plain to imaplib
https://bugs.python.org/issue45706 opened by przemub
#45709: 3.11 regression: tracing with-statement on exit from block
https://bugs.python.org/issue45709 opened by nedbat
#45710: Junction/symbolic folder access error on Windows 11
https://bugs.python.org/issue45710 opened by fstorino
#45711: Simplify the interpreter's (type, val, tb) exception represent
https://bugs.python.org/issue45711 opened by iritkatriel
#45712: Typo in "control flow" documentation
https://bugs.python.org/issue45712 opened by Andy
#45713: gcc warning when compiling Modules/expat/xmltok_ns.c
https://bugs.python.org/issue45713 opened by vamsi1281977
#45719: SubProcess stdin.flush freezes when running python interpreter
https://bugs.python.org/issue45719 opened by djp1012878
#45720: Remove shlwapi dependency on Windows
https://bugs.python.org/issue45720 opened by steve.dower
#45721: Improve error message when python shell command is entered at
https://bugs.python.org/issue45721 opened by steven.daprano
#45722: documentation missing information on objects in submodules
https://bugs.python.org/issue45722 opened by rapp.jens
#45723: Improve and simplify configure.ac checks
https://bugs.python.org/issue45723 opened by christian.heimes
#45724: Segmentation fault
https://bugs.python.org/issue45724 opened by ghost.in.the.wires
#45725: test_freeze doesn't clean up after itself
https://bugs.python.org/issue45725 opened by Mark.Shannon
#45727: Parse error when missing commas is inconsistent
https://bugs.python.org/issue45727 opened by Carl.Friedrich.Bolz
#45728: SharedMemory documentation: System V vs Posix
https://bugs.python.org/issue45728 opened by jkrupp
#45729: "history and license" link has wrong target
https://bugs.python.org/issue45729 opened by programmerjake
Most recent 15 issues with no replies (15)
==========================================
#45729: "history and license" link has wrong target
https://bugs.python.org/issue45729
#45728: SharedMemory documentation: System V vs Posix
https://bugs.python.org/issue45728
#45725: test_freeze doesn't clean up after itself
https://bugs.python.org/issue45725
#45722: documentation missing information on objects in submodules
https://bugs.python.org/issue45722
#45719: SubProcess stdin.flush freezes when running python interpreter
https://bugs.python.org/issue45719
#45713: gcc warning when compiling Modules/expat/xmltok_ns.c
https://bugs.python.org/issue45713
#45712: Typo in "control flow" documentation
https://bugs.python.org/issue45712
#45711: Simplify the interpreter's (type, val, tb) exception represent
https://bugs.python.org/issue45711
#45706: Add IMAP4.login_plain to imaplib
https://bugs.python.org/issue45706
#45701: Add tuple tests to `functools.lru_cache`
https://bugs.python.org/issue45701
#45696: "Deep-freeze": skip the marshal step by generating C code
https://bugs.python.org/issue45696
#45694: Limit the number of chained exceptions included in formatted t
https://bugs.python.org/issue45694
#45687: Infinite recursion in Pickler.persistent_id
https://bugs.python.org/issue45687
#45686: ElementTree.Element.extend: bad error message when error occur
https://bugs.python.org/issue45686
#45684: `functools.singledispatchmethod` does not define `__class_geti
https://bugs.python.org/issue45684
Most recent 15 issues waiting for review (15)
=============================================
#45727: Parse error when missing commas is inconsistent
https://bugs.python.org/issue45727
#45723: Improve and simplify configure.ac checks
https://bugs.python.org/issue45723
#45720: Remove shlwapi dependency on Windows
https://bugs.python.org/issue45720
#45711: Simplify the interpreter's (type, val, tb) exception represent
https://bugs.python.org/issue45711
#45706: Add IMAP4.login_plain to imaplib
https://bugs.python.org/issue45706
#45703: importlib.invalidate_caches() does not invalidate _NamespacePa
https://bugs.python.org/issue45703
#45701: Add tuple tests to `functools.lru_cache`
https://bugs.python.org/issue45701
#45696: "Deep-freeze": skip the marshal step by generating C code
https://bugs.python.org/issue45696
#45692: IDLE: define word/id chars in one place.
https://bugs.python.org/issue45692
#45691: Partial moving of core objects to interpreter state is incorre
https://bugs.python.org/issue45691
#45689: Custom Name for ThreadPoolExecutor
https://bugs.python.org/issue45689
#45684: `functools.singledispatchmethod` does not define `__class_geti
https://bugs.python.org/issue45684
#45680: Documentation on `GenericAlias` objects and `__class_getitem__
https://bugs.python.org/issue45680
#45677: [doc] improve sqlite3 docs
https://bugs.python.org/issue45677
#45669: An 'ascii_alphanumerics' variable is missing in the 'strings'
https://bugs.python.org/issue45669
Top 10 most discussed issues (10)
=================================
#44828: tkinter.filedialog linked with Tk 8.6.11 crashes on macOS 12 M
https://bugs.python.org/issue44828 25 msgs
#45692: IDLE: define word/id chars in one place.
https://bugs.python.org/issue45692 11 msgs
#45693: `loop.create_server` with port=0 uses different ports for ipv4
https://bugs.python.org/issue45693 9 msgs
#45669: An 'ascii_alphanumerics' variable is missing in the 'strings'
https://bugs.python.org/issue45669 8 msgs
#43158: uuid won't build when libuuid is installed in a non-standard p
https://bugs.python.org/issue43158 7 msgs
#45578: Missing tests for the dis module
https://bugs.python.org/issue45578 7 msgs
#45582: Rewrite getpath.c in Python
https://bugs.python.org/issue45582 7 msgs
#45704: string.Formatter.parse does not handle auto-numbered positiona
https://bugs.python.org/issue45704 7 msgs
#10483: http.server - what is executable on Windows
https://bugs.python.org/issue10483 6 msgs
#45615: Missing test for type of error when printing traceback for non
https://bugs.python.org/issue45615 6 msgs
Issues closed (46)
==================
#2628: ftplib Persistent data connection
https://bugs.python.org/issue2628 closed by ethan.furman
#24139: Use sqlite3 extended error codes
https://bugs.python.org/issue24139 closed by erlendaasland
#24949: Identifier lookup in a multi-level package is flakey
https://bugs.python.org/issue24949 closed by iritkatriel
#30570: issubclass segfaults on objects with weird __getattr__
https://bugs.python.org/issue30570 closed by lukasz.langa
#42064: Convert sqlite3 to multi-phase initialisation (PEP 489)
https://bugs.python.org/issue42064 closed by erlendaasland
#43969: "bad magic number" when Python 2's pyc file exists without py
https://bugs.python.org/issue43969 closed by mitchhentges
#44257: typo and verbous grammar in the grammar spec
https://bugs.python.org/issue44257 closed by pablogsal
#44319: setup openssl failed on linux
https://bugs.python.org/issue44319 closed by christian.heimes
#45335: Default TIMESTAMP converter in sqlite3 ignores UTC offset
https://bugs.python.org/issue45335 closed by lukasz.langa
#45406: inspect.getouterframes() tracebacks when $CWD does not exists
https://bugs.python.org/issue45406 closed by iritkatriel
#45457: Documentation for ssl.load_default_certs is outdated
https://bugs.python.org/issue45457 closed by lukasz.langa
#45466: Simple curl/wget-like download functionality in urllib (like h
https://bugs.python.org/issue45466 closed by eric.smith
#45484: test_pickle segfault on s390x RHEL7 LTO 3.x
https://bugs.python.org/issue45484 closed by Dennis Sweeney
#45577: Make `test_zoneinfo.py` to check all pickle protocols
https://bugs.python.org/issue45577 closed by lukasz.langa
#45581: [sqlite3] raise MemoryError if sqlite3_open_v2() returns SQLIT
https://bugs.python.org/issue45581 closed by lukasz.langa
#45588: cached_method similar to cached_property to cache with classes
https://bugs.python.org/issue45588 closed by martenlienen
#45600: First sentence in docs for os.environ
https://bugs.python.org/issue45600 closed by lukasz.langa
#45613: [sqlite3] set threadsafety attribute based on default SQLite t
https://bugs.python.org/issue45613 closed by erlendaasland
#45625: Add support for top-level await
https://bugs.python.org/issue45625 closed by asvetlov
#45633: Py_GT listed twice in Doc/extending/newtypes.rst
https://bugs.python.org/issue45633 closed by kj
#45634: [sqlite3] don't combine error checks when adding integer const
https://bugs.python.org/issue45634 closed by erlendaasland
#45642: Unable to save
https://bugs.python.org/issue45642 closed by ned.deily
#45666: Warning: "'swprintf' : format string '%s' requires an argument
https://bugs.python.org/issue45666 closed by kj
#45668: Some PGO tests are failing when building with --enable-optimiz
https://bugs.python.org/issue45668 closed by christian.heimes
#45670: New .mapping attribute is broken for some existing uses of dic
https://bugs.python.org/issue45670 closed by rhettinger
#45671: str(CancelledError()) is empty
https://bugs.python.org/issue45671 closed by lilydjwg
#45674: From Python 3.7, sre_parse.parse() do not create SubPattern in
https://bugs.python.org/issue45674 closed by serhiy.storchaka
#45678: `functools.singledispatchmethod` is missing tests (and is bugg
https://bugs.python.org/issue45678 closed by lukasz.langa
#45679: typing.Literal[True] is implicitly converted to typing.Literal
https://bugs.python.org/issue45679 closed by kj
#45682: Nicer default logging format
https://bugs.python.org/issue45682 closed by vinay.sajip
#45683: dns.asyncresolver ignores nameserver parameter
https://bugs.python.org/issue45683 closed by james2
#45685: PythonLauncher app fails run scripts on Mac BigSur
https://bugs.python.org/issue45685 closed by ned.deily
#45688: stdlib_module_names.h is missing _scproxy
https://bugs.python.org/issue45688 closed by christian.heimes
#45697: PyType_IsSubtype is doing excessive work in the common case
https://bugs.python.org/issue45697 closed by serhiy.storchaka
#45698: Error on importing getopt
https://bugs.python.org/issue45698 closed by eric.smith
#45700: Got SEGV in compilation python3.6 with GCC-11, and please rene
https://bugs.python.org/issue45700 closed by ned.deily
#45702: Python/dtoa.c requires 53 bit hardware rounding unavalable on
https://bugs.python.org/issue45702 closed by arhadthedev
#45705: |= set update scoping
https://bugs.python.org/issue45705 closed by jabowery2
#45707: Variable reassginment triggers incorrect behaviors of locals()
https://bugs.python.org/issue45707 closed by josh.r
#45708: PEP 515-style formatting with underscores does not seem to wor
https://bugs.python.org/issue45708 closed by serhiy.storchaka
#45714: test_multiprocessing_spawn hangs sometimes
https://bugs.python.org/issue45714 closed by skip.montanaro
#45715: round(2500, -3)
https://bugs.python.org/issue45715 closed by Dennis Sweeney
#45716: Confusing parsing error message when trying to use True as key
https://bugs.python.org/issue45716 closed by pablogsal
#45717: Bad link to python docs in help(_hashlib)
https://bugs.python.org/issue45717 closed by zach.ware
#45718: asyncio: MultiLoopWatcher has a race condition (Proposed work-
https://bugs.python.org/issue45718 closed by byllyfish
#45726: Documentation for `@singledispatch` and `@singledispatchmethod
https://bugs.python.org/issue45726 closed by lukasz.langa
[View Less]
1
0
Hello,
Today, an attack called "Trojan source" was revealed, where a malicious
contributor can use Unicode features (left-to-right text and homoglyphs)
to code that, when shown in an editor, will look different from how a
computer language parser will process it.
See https://trojansource.codes/, CVE-2021-42574 and CVE-2021-42694.
This is not a bug in Python.
As far as I know, the Python Security Response team reviewed the report
and decided that it should be handled in code editors, diff …
[View More]viewers,
repository frontends and similar software, rather than in the language.
I agree: in my opinion, the attack is similar to abusing any other
"gotcha" where Python doesn't parse text as a non-expert human would.
For example: `if a or b == 'yes'`, mutable default arguments, or a
misleading typo.
Nevertheless, I did do a bit of research about similar gotchas in
Python, and I'd like to publish a summary as an informational PEP,
pasted below.
> PEP: 9999
> Title: Unicode Security Considerations for Python
> Author: Petr Viktorin <encukou(a)gmail.com>
> Status: Active
> Type: Informational
> Content-Type: text/x-rst
> Created: 01-Nov-2021
> Post-History:
>
> Abstract
> ========
>
> This document explains possible ways to misuse Unicode to write Python
> programs that appear to do something else than they actually do.
>
> This document does not give any recommendations and solutions.
>
>
> Introduction
> ============
>
> Python code is written in `Unicode`_ – a system for encoding and
> handling all kinds of written language.
> While this allows programmers from all around the world to express themselves,
> it also allows writing code that is potentially confusing to readers.
>
> It is possible to misuse Python's Unicode-related features to write code that
> *appears* to do something else than what it does.
> Evildoers could take advantage of this to trick code reviewers into
> accepting malicious code.
>
> The possible issues generally can't be solved in Python itself without
> excessive restrictions of the language.
> They should be solved in code edirors and review tools
> (such as *diff* displays), by enforcing project-specific policies,
> and by raising awareness of individual programmers.
>
> This document purposefully does not give any solutions
> or recommendations: it is rather a list of things to keep in mind.
>
> This document is specific to Python.
> For general security considerations in Unicode text, see [tr36]_ and [tr39]_.
>
>
> Acknowledgement
> ===============
>
> Investigation for this document was prompted by [CVE-2021-42574],
> *Trojan Source Attacks* reported by Nicholas Boucher and Ross Anderson,
> which focuses on Bidirectional override characters in a variety of languages.
>
>
> Confusing Features
> ==================
>
> This section lists some Unicode-related features that can be surprising
> or misusable.
>
>
> ASCII-only Considerations
> -------------------------
>
> ASCII is a subset of Unicode
>
> While issues with the ASCII character set are generally well understood,
> the're presented here to help better understanding of the non-ASCII cases.
>
> Confusables and Typos
> '''''''''''''''''''''
>
> Some characters look alike.
> Before the age of computers, most mechanical typewriters lacked the keys for
> the digits ``0`` and ``1``: users typed ``O`` (capital o) and ``l``
> (lowercase L) instead. Human readers could tell them apart by context only.
> In programming language, however, distinction between digits and letters is
> critical -- and most fonts designed for programmers make it easy to tell them
> apart.
>
> Similarly, the uppercase “I” and lowercase “l” can look similar in fonts
> designed for human languages, but programmers' fonts make them noticeably
> different.
>
> However, what is “noticeably” different always depend on the context.
> Humans tend to ignore details in longer identifiers: the variable name
> ``accessibi1ity_options`` can still look indistinguishable from
> ``accessibility_options``, while they are distinct for the compiler.
>
> The same can be said for plain typos: most humans will not notice the typo in
> ``responsbility_chain_delegate``.
>
> Control Characters
> ''''''''''''''''''
>
> Python generally considers all ``CR`` (``\r``), ``LF`` (``\n``), and ``CR-LF``
> pairs (``\r\n``) as an end of line characters.
> Most code editors do as well, but there are editors that display “non-native”
> line endings as unknown characters (or nothing at all), rather than ending
> the line, displaying this example::
>
> # Don't call this function:
> fire_the_missiles()
>
> as a harmless comment like::
>
> # Don't call this function:⬛fire_the_missiles()
>
> CPython treats the control character NUL (``\0``) as end of input,
> but many editors simply skip it, possibly showing code that Python will not
> run as a regular part of a file.
>
> Some characters can be used to hide/overwrite other characters when source is
> listed in common terminals:
>
> * BS (``\b``, Backspace) moves the cursor back, so the character after it
> will overwrite the character before.
> * CR (``\r``, carriage return) moves the cursor to the start of line,
> subsequent characters overwrite the start of the line.
> * DEL (``\x7F``) commonly initiates escape codes which allow arbitrary
> control of the terminal.
>
>
> Confusable Characters in Identifiers
> ------------------------------------
>
> Python allows characters of all any scripts – Latin letters to ancient Egyptian
> hieroglyphs – in identifiers (such as variable names).
> See :pep:`3131` for details and rationale.
> Only “letters and numbers” are allowed (see `Identifiers and keywords`_
> for details), so while ``γάτα`` is a valid Python identifier, ``🐱`` is not.
> Non-printing control characters are also not allowed.
>
> However, within the allowed set there is a large number of “confusables”.
> For example, the uppercase versions of the Latin `b`, Greek `β` (Beta), and
> Cyrillic `в` (Ve) often look identical: ``B``, ``Β`` and ``В``, respectively.
>
> This allows identifiers that look the same to humans, but not to Python.
> For example, all of the following are distinct identifiers:
>
> * ``scope`` (Latin, ASCII-only)
> * ``scоpe`` (wih a Cyrillic `о`)
> * ``scοpe`` (with a Greek `ο`)
> * ``ѕсоре`` (all Cyrillic letters)
>
> Additionally, some letters can look like non-letters:
>
> * The letter for the Hawaiian *ʻokina* looks like an apostrophe;
> ``ʻHelloʻ`` is a Python identifier, not a string.
> * The East Asian symbol for *ten* looks like a plus sign,
> so ``十= 10`` is a complete Python statement.
>
> .. note::
>
> The converse also applies – some symbols look like letters – but since
> Python does not allow arbitrary symbols in identifiers, this is not an
> issue.
>
>
> Confusable Digits
> ------------------
>
> Numeric literals in Python only use the ASCII digits 0-9 (and non-digits such
> as ``.`` or ``e``).
>
> However, when numbers are converted from strings, such as in the ``int`` and
> ``float`` constructors or by the ``str.format`` method, any decimal digit
> can be used. For example ``߅`` (``NKO DIGIT FIVE``) or ``௫``
> (``TAMIL DIGIT FIVE``) work as the digit ``5``.
>
> Some scripts include digits that look similar to ASCII ones, but have a
> different value. For example::
>
> >>> int('৪୨')
> 42
> >>> '{٥}'.format('zero', 'one', 'two', 'three', 'four', 'five')
> five
>
>
> Bidirectional Text
> ------------------
>
> Some scripts, such as Hebrew or Arabic, are written right-to-left.
> Phrases in such scripts interact with nearby text in ways that can be
> surprising to people who aren't familiar with these writing systems and their
> computer representation.
>
> The exact process is complicated, and explained in Unicode® Standard Annex #9,
> "Unicode Bidirectional Algorithm".
>
> Some surprising examples include:
>
> * In the statement ``ערך = 23``, the variable ``ערך`` is set to the integer 23.
>
> * In the statement ``قيمة = ערך``, the variable ``قيمة`` is set
> to the value of ``ערך``.
>
> * In the statement ``قيمة - (ערך ** 2)``, the value of ``ערך`` is squared and
> then subtracted from ``قيمة``.
> The *opening* parenthesis is displayed as ``)``.
>
> * In the following, the second line is the same as first, except
> ``A`` is replaced by the Hebrew ``א``. Both assign a 100-character string to
> the variable ``s``.
> Note how the symbols and numbers between Hebrew characters is shown in
> reverse order::
>
> s = "A" * 100 # "A" is assigned
>
> s = "א" * 100 # "א" is assigned
>
>
> Bidirectional Marks, Embeddings, Overrides and Isolates
> -------------------------------------------------------
>
> The rules for determining the direction of text do not always yield the
> intended results, so Unicode provides several ways to alter it.
>
> The most basic are **directional marks**, which are invisible but affect text
> as a left-to-right (or right-to-left) character would.
> Following with the example above, in the next example the ``A``/``א`` is
> replaced by the Latin ``x`` followed or preceded by a
> right-to-left mark (``U+200F``). This assigns a 200-character string to ``s``
> (100 copies of `x` interspersed with 100 invisible marks)::
>
> s = "x" * 100 # "x" is assigned
>
> The directional **embedding**, **override** and **isolate** characters
> are also invisible, but affect the ordering of all text after them until either
> ended by a dedicated character, or until the end of line.
> (Unicode specifies the effect to last until the end of a “paragraph” (see [tr9]_),
> but allows tools to interpret newline characters as paragraph ends
> (see [u5.8]_). Most code editors and terminals do so.)
>
> These characters essentially allow arbitrary reordering of the text that
> follows them. Python only allows them in strings and comments, which does limit
> their potential (especially in combination with the fact that Python's comments
> always extend to the end of a line), but it doesn't render them harmless.
>
>
> Normalizing identifiers
> -----------------------
>
> Python strings are collections of *Unicode codepoints*, not “characters”.
>
> For reasons like compatibility with earlier encodings, Unicode often has
> several ways to encode what is essentially a single “character”.
> For example, all are these different ways of writing ``Å`` as a Python string,
> each of which is unequal to the others.
>
> * ``"\N{LATIN CAPITAL LETTER A WITH RING ABOVE}"`` (1 codepoint)
> * ``"\N{LATIN CAPITAL LETTER A}\N{COMBINING RING ABOVE}"`` (2 codepoints)
> * ``"\N{ANGSTROM SIGN}"`` (1 codepoint, but different)
>
> For another example, the ligature ``fi`` has a dedicated Unicode codepoint,
> even though it has the same meaning as the two letters ``fi``.
>
> Also, common letters frequently have several distinct variations.
> Unicode provides them for contexts where the difference has some semantic
> meaning, like mathematics. For example, some variations of ``n`` are:
>
> * ``n`` (LATIN SMALL LETTER N)
> * ``𝐧`` (MATHEMATICAL BOLD SMALL N)
> * ``𝘯`` (MATHEMATICAL SANS-SERIF ITALIC SMALL N)
> * ``n`` (FULLWIDTH LATIN SMALL LETTER N)
> * ``ⁿ`` (SUPERSCRIPT LATIN SMALL LETTER N)
>
> Unicode includes alorithms to *normalize* variants like these to a single
> form, and Python identifiers are normalized.
> (There are several normal forms; Python uses ``NFKC``.)
>
> For example, ``xn`` and ``xⁿ`` are the same identifier in Python::
>
> >>> xⁿ = 8
> >>> xn
> 8
>
> … as is ``fi`` and ``fi``, and as are the different ways to encode ``Å``.
>
> This normalization applies *only* to identifiers, however.
> Functions that treat strings as identifiers, such as ``getattr``,
> do not perform normalization::
>
> >>> class Test:
> ... def finalize(self):
> ... print('OK')
> ...
> >>> Test().finalize()
> OK
> >>> Test().finalize()
> OK
> >>> getattr(Test(), 'finalize')
> Traceback (most recent call last):
> ...
> AttributeError: 'Test' object has no attribute 'finalize'
>
> This also applies when importing:
>
> * ``import finalization`` performs normalization, and looks for a file
> named ``finalization.py`` (and other ``finalization.*`` files).
> * ``importlib.import_module("finalization")`` does not normalize,
> so it looks for a file named ``finalization.py``.
>
> Some filesystems independently apply normalization and/or case folding.
> On some systems, ``finalization.py``, ``finalization.py`` and
> ``FINALIZATION.py`` are three distinct filenames; on others, some or all
> of these can name the same file.
>
>
> Source Encoding
> ---------------
>
> The encoding of Python source files is given by a specific regex on the first
> two lines of a file, as per `Encoding declarations`_.
> This mechanism is very liberal in what it accepts, and thus easy to obfuscate.
>
> This can be misused in combination with Python-specific special-purpose
> encodings (see `Text Encodings`_).
> For example, with ``encoding: unicode_escape``, characters like
> quotes or braces can be hidden in an (f-)string, with many tools (syntax
> highlighters, linters, etc.) considering them part of the string.
> For example::
>
> # For writing Japanese, you don't need an editor that supports
> # UTF-8 source encoding: unicode_escape sequences work just as well.
>
> import os
>
> message = '''
> This is "Hello World" in Japanese:
> \u3053\u3093\u306b\u3061\u306f\u7f8e\u3057\u3044\u4e16\u754c
>
> This runs `echo WHOA` in your shell:
> \u0027\u0027\u0027\u002c\u0028\u006f\u0073\u002e
> \u0073\u0079\u0073\u0074\u0065\u006d\u0028
> \u0027\u0065\u0063\u0068\u006f\u0020\u0057\u0048\u004f\u0041\u0027
> \u0029\u0029\u002c\u0027\u0027\u0027
> '''
>
>
> Open Issues
> ===========
>
> We should probably write and publish:
>
> * Recommendations for Text Editors and Code Tools
> * Recommendations for Programmers and Teams
> * Possible Improvements in Python
>
>
> References
> ==========
>
> .. _Unicode: https://home.unicode.org/
> .. _`Encoding declarations`: https://docs.python.org/3/reference/lexical_analysis.html#encoding-declarat…
> .. _`Identifiers and keywords`: https://docs.python.org/3/reference/lexical_analysis.html#identifiers
> .. _`Text Encodings`: https://docs.python.org/3/library/codecs.html#text-encodings
> .. [u5.8] http://www.unicode.org/versions/Unicode14.0.0/ch05.pdf#G10213
> .. [tr9] http://www.unicode.org/reports/tr9/
> .. [tr36] Unicode Technical Report #36: Unicode Security Considerations
> http://www.unicode.org/reports/tr39/
> .. [tr39] Unicode® Technical Standard #39: Unicode Security Mechanisms
> http://www.unicode.org/reports/tr39/
> .. [CVE-2021-42574] CVE-2021-42574
> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-42574
>
>
> Copyright
> =========
>
> This document is placed in the public domain or under the
> CC0-1.0-Universal license, whichever is more permissive.
>
>
>
> ..
> Local Variables:
> mode: indented-text
> indent-tabs-mode: nil
> sentence-end-double-space: t
> fill-column: 70
> coding: utf-8
> End:
>
[View Less]
13
44
As you might have noticed, 3.9.8 was scheduled for release on Monday. This didn't happen yet.
There's a bunch of ongoing work fixing Tcl/Tk problems. macOS Monterey got released with a new incompatible Tcl/Tk version, some fixes were required for tkinter compatibility. Details in https://bugs.python.org/issue44828 <https://bugs.python.org/issue44828>. This is now a fixed issue (thanks, Ned!) but we're going through some more ttk fixes discovered in failing buildbots, which were sadly …
[View More]unrelated to the Tcl/Tk version bump. In particular, there's a ttk refleak that was discovered by the Windows 8.1 refleak buildbot which turned out to be a widespread issue (other refleak buildbots were simply skipping GUI tests). Example failure: https://buildbot.python.org/all/#/builders/6/builds/168 <https://buildbot.python.org/all/#/builders/6/builds/168>
I'm fixing the macOS ttk test failures and the refleak now to unblock the 3.9.8 release. I should be done with this tomorrow. We'll go ahead with the release then. Pablo decided to wait for me with 3.11a2 since the problems are fixing are also happening in `main` (and 3.10).
We'll keep you posted, cheers!
--
Łukasz Langa
CPython Developer in Residence
Python Software Foundation
[View Less]
1
0
See the latest changes, which are mostly a (hopefully) improved abstract, better tables, and some slight rewordings.
Feedback welcome!
-----------------------------------
PEP: 663
Title: Standardizing Enum str(), repr(), and format() behaviors
Version: $Revision$
Last-Modified: $Date$
Author: Ethan Furman <ethan(a)stoneleaf.us>
Discussions-To: python-dev(a)python.org
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 23-Feb-2013
Python-Version: 3.11
Post-History: 20-…
[View More]Jul-2021, 02-Nov-2021
Resolution:
Abstract
========
Update the ``repr()``, ``str()``, and ``format()`` of the various Enum types
to better match their intended purpose. For example, ``IntEnum`` will have
its ``str()`` change to match its ``format()``, while a user-mixed int-enum
will have its ``format()`` match its ``str()``. In all cases, an enum's
``str()`` and ``format()`` will be the same (unless the user overrides
``format()``).
Add a global enum decorator which changes the ``str()`` and ``repr()`` (and
``format()``) of the decorated enum to be a valid global reference: i.e.
``re.IGNORECASE`` instead of ``<RegexFlag.IGNORECASE: 2>``.
Motivation
==========
Having the ``str()`` of ``IntEnum`` and ``IntFlag`` not be the value causes
bugs and extra work when replacing existing constants.
Having the ``str()`` and ``format()`` of an enum member be different can be
confusing.
The addition of ``StrEnum`` with its requirement to have its ``str()`` be its
``value`` is inconsistent with other provided Enum's ``str``.
The iteration of ``Flag`` members, which directly affects their ``repr()``, is
inelegant at best, and buggy at worst.
Rationale
=========
Enums are becoming more common in the standard library; being able to recognize
enum members by their ``repr()``, and having that ``repr()`` be easy to parse, is
useful and can save time and effort in understanding and debugging code.
However, the enums with mixed-in data types (``IntEnum``, ``IntFlag``, and the new
``StrEnum``) need to be more backwards compatible with the constants they are
replacing -- specifically, ``str(replacement_enum_member) == str(original_constant)``
should be true (and the same for ``format()``).
IntEnum, IntFlag, and StrEnum should be as close to a drop-in replacement of
existing integer and string constants as is possible. Towards that goal, the
``str()`` output of each should be its inherent value; e.g. if ``Color`` is an
``IntEnum``::
>>> Color.RED
<Color.RED: 1>
>>> str(Color.RED)
'1'
>>> format(Color.RED)
'1'
Note that ``format()`` already produces the correct output, only ``str()`` needs
updating.
As much as possible, the ``str()``, ``repr()``, and ``format()`` of enum members
should be standardized across the standard library. However, up to Python 3.10
several enums in the standard library have a custom ``str()`` and/or ``repr()``.
The ``repr()`` of Flag currently includes aliases, which it should not; fixing that
will, of course, already change its ``repr()`` in certain cases.
Specification
=============
There a three broad categories of enum usage:
- simple: ``Enum`` or ``Flag``
a new enum class is created with no data type mixins
- drop-in replacement: ``IntEnum``, ``IntFlag``, ``StrEnum``
a new enum class is created which also subclasses ``int`` or ``str`` and uses
``int.__str__`` or ``str.__str__``
- user-mixed enums and flags
the user creates their own integer-, float-, str-, whatever-enums instead of
using enum.IntEnum, etc.
There are also two styles:
- normal: the enumeration members remain in their classes and are accessed as
``classname.membername``, and the class name shows in their ``repr()`` and
``str()`` (where appropriate)
- global: the enumeration members are copied into their module's global
namespace, and their module name shows in their ``repr()`` and ``str()``
(where appropriate)
Some sample enums::
# module: tools.py
class Hue(Enum): # or IntEnum
LIGHT = -1
NORMAL = 0
DARK = +1
class Color(Flag): # or IntFlag
RED = 1
GREEN = 2
BLUE = 4
class Grey(int, Enum): # or (int, Flag)
BLACK = 0
WHITE = 1
Using the above enumerations, the following two tables show the old and new
output (blank cells indicate no change):
+--------+------------------------+-----------------+------------+-----------------------+
| style | category | enum repr() | enum str() | enum format() |
+--------+-------------+----------+-----------------+------------+-----------------------+
| normal | simple | 3.10 | | | |
| | +----------+-----------------+------------+-----------------------+
| | | new | | | |
| +-------------+----------+-----------------+------------+-----------------------+
| | user mixed | 3.10 | | | 1 |
| | +----------+-----------------+------------+-----------------------+
| | | new | | | Grey.WHITE |
| +-------------+----------+-----------------+------------+-----------------------+
| | int drop-in | 3.10 | | Hue.LIGHT | |
| | +----------+-----------------+------------+-----------------------+
| | | new | | -1 | |
+--------+-------------+----------+-----------------+------------+-----------------------+
| global | simple | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT |
| | +----------+-----------------+------------+-----------------------+
| | | new | tools.LIGHT | LIGHT | LIGHT |
| +-------------+----------+-----------------+------------+-----------------------+
| | user mixed | 3.10 | <Grey.WHITE: 1 | Grey.WHITE | Grey.WHITE |
| | +----------+-----------------+------------+-----------------------+
| | | new | tools.WHITE | WHITE | WHITE |
| +-------------+----------+-----------------+------------+-----------------------+
| | int drop-in | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | |
| | +----------+-----------------+------------+-----------------------+
| | | new | tools.LIGHT | -1 | |
+--------+-------------+----------+-----------------+------------+-----------------------+
+--------+------------------------+-----------------------+------------------------+-----------------------+
| style | category | flag repr() | flag str() | flag format() |
+--------+-------------+----------+-----------------------+------------------------+-----------------------+
| normal | simple | 3.10 | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN |
| +-------------+----------+-----------------------+------------------------+-----------------------+
| | user mixed | 3.10 | <Grey.WHITE: 1> | | 1 |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | <Grey(1): WHITE> | | Grey.WHITE |
| +-------------+----------+-----------------------+------------------------+-----------------------+
| | int drop-in | 3.10 | <Color.RED|GREEN: 3> | Color.RED|GREEN | |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | <Color(3): RED|GREEN> | 3 | |
+--------+-------------+----------+-----------------------+------------------------+-----------------------+
| global | simple | 3.10 | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN |
| +-------------+----------+-----------------------+------------------------+-----------------------+
| | user mixed | 3.10 | <Grey.WHITE: 1> | Grey.WHITE | 1 |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | tools.WHITE | WHITE | WHITE |
| +-------------+----------+-----------------------+------------------------+-----------------------+
| | int drop-in | 3.10 | <Color.RED|GREEN: 3> | Color.RED|GREEN | |
| | +----------+-----------------------+------------------------+-----------------------+
| | | new | tools.RED|tools.GREEN | 3 | |
+--------+-------------+----------+-----------------------+------------------------+-----------------------+
These two tables show the final result:
+--------+-------------+-----------------+------------+-----------------------+
| style | category | enum repr() | enum str() | enum format() |
+--------+-------------+-----------------+------------+-----------------------+
| normal | simple | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT |
| +-------------+-----------------+------------+-----------------------+
| | user mixed | <Grey.WHITE: 1> | Grey.WHITE | Grey.WHITE |
| +-------------+-----------------+------------+-----------------------+
| | int drop-in | <Hue.LIGHT: -1> | -1 | -1 |
+--------+-------------+-----------------+------------+-----------------------+
| global | simple | tools.LIGHT | LIGHT | LIGHT |
| +-------------+-----------------+------------+-----------------------+
| | user mixed | tools.WHITE | WHITE | WHITE |
| +-------------+-----------------+------------+-----------------------+
| | int drop-in | tools.LIGHT | -1 | -1 |
+--------+-------------+-----------------+------------+-----------------------+
+--------+-------------+-----------------------+------------------------+-----------------------+
| style | category | flag repr() | flag str() | flag format() |
+--------+-------------+-----------------------+------------------------+-----------------------+
| normal | simple | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN |
| +-------------+-----------------------+------------------------+-----------------------+
| | user mixed | <Grey(1): WHITE> | Grey.WHITE | Grey.WHITE |
| +-------------+-----------------------+------------------------+-----------------------+
| | int drop-in | <Color(3): RED|GREEN> | 3 | 3 |
+--------+-------------+-----------------------+------------------------+-----------------------+
| global | simple | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN |
| +-------------+-----------------------+------------------------+-----------------------+
| | user mixed | tools.WHITE | WHITE | WHITE |
| +-------------+-----------------------+------------------------+-----------------------+
| | int drop-in | tools.RED|tools.GREEN | 3 | 3 |
+--------+-------------+-----------------------+------------------------+-----------------------+
As can be seen, ``repr()`` is primarily affected by whether the members are
global, while ``str()`` is affected by being global or by being a drop-in
replacement, with the drop-in replacement status having a higher priority.
Also, the basic ``repr()`` and ``str()`` have changed for flags as the old
style was flawed.
Backwards Compatibility
=======================
Backwards compatibility of stringified objects is not guaranteed across major
Python versions, and there will be backwards compatibility breaks where
software uses the ``repr()``, ``str()``, and ``format()`` output of enums in
tests, documentation, data structures, and/or code generation.
Normal usage of enum members will not change: ``re.ASCII`` can still be used
as ``re.ASCII`` and will still compare equal to ``256``.
If the previous output needs to be maintained, for example to ensure
compatibility between different Python versions, software projects will need to
create their own enum base class with the appropriate methods overridden.
Note that by changing the ``str()`` of the drop-in category, we will actually
prevent future breakage when ``IntEnum``, et al, are used to replace existing
constants.
Copyright
=========
This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
[View Less]
2
1