Mailman 3 October 2021 - Python-Dev

Documenting Python versioning and stability expectations
by Petr Viktorin Oct. 16, 2021

Oct. 16, 2021

Hello, I heard not everyone is Discourse, so I'm re-posting here as well. Information about <topic> is current scattered over the FAQs, active PEPs (387, 602), an active-but-severely-outdated PEP (6) and the Devguide. I would like to consolidate as much of this as possible into user-facing reference documentation. * Does that sound like a good idea? * Where in the docs should such a page live? Here is a draft of the docs; if added they’d be accompanied by more changes: * The relevant FAQs would be replaced by links to these docs * PEP 6 would be retired in favor of 602, 387 and these docs * Parts of PEP 387/602 might be replaced by links here (or this document should link to the PEP rather than duplicating it) * The examples of “backwards-incompatible APIs” from PEP 387 would only live in one place, and be linked from the other. * C-API stability docs should link back to this more general document. (Details specific to the C-API shoudl still be there.) Most of this is, hopefully, just capturing existing tribal knowledge, but: * the “Unstable API” section contains several additional examples that PEP 387 doesn’t have. I’m proposing them as an update to the PEP; all of this will need SC approval anyway. * “Most planned changes (such as removal of deprecated features) are done in alpha releases.” – this is also a newly proposed rule; I intend to formalize it further for the devguide. Combined with our deprecation periods, this rule would allow more time for testing the actual effect of removals. A rendered (and up-to-date) version is at: https://github.com/encukou/cpython/blob/cpython-version-docs/Doc/stability-… ------------------------------------------------ .. _python-versioning: ============================================== Python's Versioning and Stability Expectations ============================================== Python Versions =============== The Python language is developed together with its reference :ref:`implementation <implementations>`, *CPython*. Both share the same release schedule and versioning scheme. Production-ready Python versions numbered with three numbers, ``major.minor.micro``. * New *major* versions are exceptional, and are planned very long in advance. * New *minor* versions are feature releases; they get released annually. * New *micro* versions are *bugfix* releases, which get released roughly every 2 months for 5 years after a minor release; or *security* releases which are made irregularly afterwards. We also publish non-final *pre-release* versions with an additional qualifier: *Alpha* (``a``), *Beta* (``b``) and *release candidates* (``rc``). These versions are not production use; they're aimed at testers and maintainers of third-party libraries. The version number is combined into a single string, for example: +----------------------+-------+-------+-------+----------------------------+ | Version | Major | Minor | Micro | Prerelease | +======================+=======+=======+=======+============================+ | Python ``3.6.3`` | 3 | 6 | 3 | Final (production-ready) | +----------------------+-------+-------+-------+----------------------------+ | Python ``3.7.0a3`` | 3 | 7 | 0 | Third *alpha* | +----------------------+-------+-------+-------+----------------------------+ | Python ``3.9.4b1`` | 3 | 9 | 4 | First *beta* | +----------------------+-------+-------+-------+----------------------------+ | Python ``2.7.14rc2`` | 2 | 7 | 14 | Second *release candidate* | +----------------------+-------+-------+-------+----------------------------+ When discussing features that do not change in *micro* or *minor* releases, or ones that are new in `x.y.0` or `x.0.0` versions, it is common to only specify the relevant numbers: +-----------------+-------+-------+-------+ | Version | Major | Minor | Micro | +=================+=======+=======+=======+ | Python ``3.10`` | 3 | 10 | any | +-----------------+-------+-------+-------+ | Python ``3`` | 3 | any | any | +-----------------+-------+-------+-------+ See also the documentation for :data:`sys.version_info`, :data:`sys.hexversion`, :data:`sys.version`, and :ref:`apiabiversion`, which expose version numbers in different formats. .. _python-releases: Python Releases =============== All releases, including pre-releases, are available from https://www.python.org/downloads/. New releases are announced on the comp.lang.python and comp.lang.python.announce newsgroups and on the Python home page at `python.org`_; an RSS feed of news is available. .. _python.org: https://python.org .. _python-stability: Versioning details and Stability Expectations ============================================= This section documents stability expectations for the various types of Python releases. It is intended for users of Python, that is, general programmers and library maintainers. For developers *of* Python, this topic is covered in the `Development Cycle`_ section of the devguide and in :pep:`387` -- *Backwards Compatibility Policy*. The *schedules* for specific minor-version releases are covered in the `Status of Python branches`_ and `End-of-life branches`_ sections of the devguide, where they're updated regularly. For details on stability of Python's C API, see :ref:`stable`. .. note:: The “stability expectations” covered here are *goals*. Sometimes, Python will contain bugs in the form of unintentional backwards incompatible changes. Sometimes, a carefully considered exception to the rules is made. Python relies on people testing with pre-releases and reporting issues to find edge cases of backwards incompatibilities: if backwards compatibility is important to you, consider testing your codebase with Python pre-releases. .. _Development Cycle: https://devguide.python.org/devcycle/#devcycle .. _Status of Python branches: https://devguide.python.org/#status-of-python-branches .. _End-of-life branches: https://devguide.python.org/devcycle/#end-of-life-branches Major Versions -------------- When Python 3.0.0 was released, most programs for Python 2 needed to be adjusted for compatibility. The transition took several decades. There are currently no plans to release Python 4. The expectation is that the transition to Python 4.0, if and when it happens, will be similar to the transition to a new minor version. Minor Versions -------------- Minor versions usually contain many new features and bugfixes. All changes are collected in a :ref:`What's New <whatsnew-index>` document. A new minor release may not be fully compatible with its predecessor. Incompatibilities -- changes that make preexisting code cease to comparatively function -- are generally only made after a *deprecation period*: code whose behavior will change will emit :exc:`DeprecationWarning` for at least two minor releases before being removed. However, in extreme cases (e.g. for dangerously broken or insecure features) the deprecation period may be skipped. In new minor versions, Python may remove support for little-used platforms. See :pep:`11` for more details and the process to get a platform re-supported. .. note:: Despite similarities, Python does not use the popular `semantic versioning`_ scheme, which was published several decades after Python. .. _semantic versioning: https://semver.org/ Micro Versions -------------- A new micro version marks *bugfix* and *security* releases. These releases are managed for stability; only fixes for known problems are included in them, and Python's interfaces do not change in new micro versions. Generally, it is enough for third-party libraries to test with one release of a minor version -- ideally the latest one. For example, a library tested with Python 3.5.10 may reasonably claim to be compatible with Python 3.5 in general. Bugfix Releases ............... For about 18 months (1½ years) after a new minor version, *bugfix releases* are made approximately every two months. Bugfix releases contain sources and installers for Windows and macOS. Backwards-incompatible changes are rare in bugfix releases, but sometimes necessary for to fix serious bugs or to make Python work as documented. Security Releases ................. For forty two months (3½ years) after the bugfix period is over, *security releases* are made on an as-needed basis (no fixed cadence). For these, the micro version is incremented just like with bugfix releases. Security releases only fix exploitable issues like crashes and possibilities of privilege escalation. Security releases are *source-only*: Windows and macOS installers for them are not available from `python.org`_, which means that users of these binaries should upgrade after the bugfix period ends. (Other distributors of Python will have their own support periods.) End-of-life ........... Five years after the initial release of a particular minor version, a final security release is published and the minor version reaches *End-of-life*. No more changes at all are made to that minor version. .. _python-prereleases: Pre-releases ------------ Several months before a new minor version, and sometimes before a new micro version, snapshots of the development branch are released as *pre-releases*. Rough planned dates for pre-releases are recorded in Release Schedule PEPs, as listed in the `Status of Python branches`_ section of the devguide. Alpha versions .............. Alpha versions give the community a chance to test a very early version of the upcoming release and report any issues. Most planned changes (such as removal of deprecated features) are done in alpha releases. Early new features are included as well, but additional new features can be added to Python up until the first Beta release. Alpha versions are unstable. (See Beta versions below for more details.) Beta versions ............. After the first Beta release, no new features are added, and development focuses on fixing bugs and improving documentation. Maintainers of third-party libraries are encouraged to test their code with Beta releases, so any bugs or unforeseen issues can be resolved as early as possible. New features can be *removed* from the upcoming release in the Beta period. Beta versions are unstable. There are no backwards compatibility guarantees between beta versions. For example, bytecode cache (`.pyc`) files may not be compatible and the :ref:`ABI <stable>` for C-API extensions may still change. Sharing :mod:`virtual environments <venv>` (and installed libraries in general) between different beta versions is not a good idea. Release Candidates .................. The goal of Release Candidates (RCs) is to enable final testing and prepare the wider Python ecosystem for the upcoming release. Only serious bugs are fixed in RCs (aside from improvements to documentation and internal tests). Ideally, there would be no changes between a RC and the final release. Release Candidates are *stable*; the bar for changes in new RCs is higher than for new micro versions. Third-party libraries that release for each minor version can and should release in the RC period, so that users find the library installable when the final version comes out. For example, a build of `Numpy`_ for `3.10` can and should be published with `3.10.0rc1`. The changes between `3.10.0rc1` and `3.10.0` will be smaller than between `3.10.0` and `3.10.1`. .. _NumPy: https://numpy.org/ Unstable API ------------ The following are *not* considered stable, and may change at any time -- even in new micro versions (although that doesn't happen without a strong reason): * Anything (functions, classes, modules, attributes, methods, C-API names and types, etc.) with a name prefixed by "_", except :ref:`special names <specialnames>`). * Anything documented publicly as being private. * Imported modules (unless explicitly documented as part of the public API; e.g. if the ``spam`` module imports the ``bacon`` module, it does not automatically mean ``spam.bacon`` is part of the public API unless it is documented as such). * Inheritance patterns of internal classes. * Test suites. (Anything in the ``test`` package or ``test`` sub-packages.) * API that is explicitly documented as :term:`provisional <provisional API>`. * Features enabled by :ref:`future statements <future>`. * The exact text and formatting of error messages and tracebacks. * String representations of objects (results of ``str()`` and ``repr()``), unless documented. * Exact types: any type may be replaced with a subtype (e.g. `FileNotFoundError` can be raised where `OSError` was raised previously). * Exact output of parsing, serialization, compression, etc. For example, Zip, Pickle or ``*.pyc`` files produced from the same data may not be bit-by-bit identical, though they should be *functionally* identical. All of these items will also often differ across :ref:`implementations <implementations>` of the Python language. Portable code should not rely on these details.

4 3

Summary of Python tracker Issues
by Python tracker Oct. 15, 2021

Oct. 15, 2021

ACTIVITY SUMMARY (2021-10-08 - 2021-10-15) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7401 (+27) closed 49822 (+50) total 57223 (+77) Open issues with patches: 2925 Issues opened (50) ================== #35081: Move internal headers to Include/internal/ https://bugs.python.org/issue35081 reopened by vstinner #35134: Add a new Include/cpython/ subdirectory for the "CPython API" https://bugs.python.org/issue35134 reopened by vstinner #42248: Raised exception in Enum keeping user objects alive unnecessar https://bugs.python.org/issue42248 reopened by pitrou #44559: Enum: revert to 3.9 https://bugs.python.org/issue44559 reopened by pitrou #45249: Update doctect SyntaxErrors for location range https://bugs.python.org/issue45249 reopened by pablogsal #45414: pathlib.Path.parents negative indexing is wrong for absolute p https://bugs.python.org/issue45414 opened by josh.r #45415: Assert oparg < INSTR_OFFSET() https://bugs.python.org/issue45415 opened by steve.dower #45417: Enum creation non-linear in the number of values https://bugs.python.org/issue45417 opened by olliemath #45420: Python 3.10 final installation failure https://bugs.python.org/issue45420 opened by wyz23x2 #45423: SSL SNI varies when host contains port number https://bugs.python.org/issue45423 opened by hexf #45425: There is an error in the Chinese documentation of contextlib.A https://bugs.python.org/issue45425 opened by laxtiz #45427: importlib.readers.MultiplexedPath https://bugs.python.org/issue45427 opened by daveraja #45429: [Windows] time.sleep() should use CREATE_WAITABLE_TIMER_HIGH_R https://bugs.python.org/issue45429 opened by vstinner #45431: [C API] Rename CFrame or hide it to only export names starting https://bugs.python.org/issue45431 opened by vstinner #45435: delete misleading faq entry about atomic operations https://bugs.python.org/issue45435 opened by graingert #45436: test_tk.test_configure_type() failed on x86 Gentoo Non-Debug w https://bugs.python.org/issue45436 opened by vstinner #45438: inspect not capturing type annotations created by __class_geti https://bugs.python.org/issue45438 opened by rhettinger #45442: Update `Virtual Environment` tutorial https://bugs.python.org/issue45442 opened by shivnaren #45443: 'ThreadPoolExecutor' object has no attribute 'map' https://bugs.python.org/issue45443 opened by zohim #45444: test.test_concurrent_futures fail in x86_ 64 architecture https://bugs.python.org/issue45444 opened by sxt1001 #45445: Fail if an invalid -X option is provided https://bugs.python.org/issue45445 opened by pablogsal #45446: Add a way to hide fields in dataclasses https://bugs.python.org/issue45446 opened by formigacomcaimbra #45447: IDLE: Support syntax highlighting for .pyi stub files https://bugs.python.org/issue45447 opened by AlexWaygood #45449: Document that collections.abc types are subscriptable https://bugs.python.org/issue45449 opened by FFY00 #45450: Improve syntax error for parenthesized arguments https://bugs.python.org/issue45450 opened by pablogsal #45451: IDLE Shell GUI - remove window border https://bugs.python.org/issue45451 opened by primexx #45452: Support crash tolerance feature for gdbm module https://bugs.python.org/issue45452 opened by corona10 #45455: Fill func.__doc__ lazily https://bugs.python.org/issue45455 opened by methane #45457: Documentation for ssl.load_default_certs is outdated https://bugs.python.org/issue45457 opened by LincolnPuzey #45459: Limited API support for Py_buffer https://bugs.python.org/issue45459 opened by christian.heimes #45460: distutils.sysconfig.get_python_lib() does not respect sysconfi https://bugs.python.org/issue45460 opened by hroncok #45463: Documentation inconsistency on the number of identifiers allow https://bugs.python.org/issue45463 opened by lucach #45464: TypeError when inheriting from both OSError and AttributeError https://bugs.python.org/issue45464 opened by marmarek #45466: Simple curl/wget-like download functionality in urllib (like h https://bugs.python.org/issue45466 opened by tom.pohl #45468: Add support for preloading a Python script https://bugs.python.org/issue45468 opened by Gabriele Tornetta #45472: Add public C API for partial "unicode-escape" and "raw-unicode https://bugs.python.org/issue45472 opened by serhiy.storchaka #45473: Enum: add "name" and "value" keywords to call syntax https://bugs.python.org/issue45473 opened by aekoch #45475: gzip fails to read a gzipped file (ValueError: readline of clo https://bugs.python.org/issue45475 opened by minstrelofc #45476: [C API] Disallow using PyFloat_AS_DOUBLE() as l-value https://bugs.python.org/issue45476 opened by vstinner #45477: configure script cannot detect float word ordering on linux https://bugs.python.org/issue45477 opened by sourish.basu #45478: Send a patch to libexpat to solve _POSIX_C_SOURCE issue. https://bugs.python.org/issue45478 opened by corona10 #45480: Missing link(s) to the "Python Module Index" page https://bugs.python.org/issue45480 opened by mark #45483: pure Python class that has been registered as a `collections.a https://bugs.python.org/issue45483 opened by GalaxySnail #45484: test_pickle segfault on s390x RHEL7 LTO 3.x https://bugs.python.org/issue45484 opened by vstinner #45485: datetime.strptime issues message attribute of type 'NoneType' https://bugs.python.org/issue45485 opened by f.de.kruijf #45486: Stop using internal APIs in _xxsubinterpretersmodule.c. https://bugs.python.org/issue45486 opened by eric.snow #45487: SSLEOFError regression with certain servers in Python 3.10 https://bugs.python.org/issue45487 opened by nyuszika7h #45488: Powershell Commands no Longer Recognised https://bugs.python.org/issue45488 opened by julienbloxerk #45489: ForwardRef does not support | operator https://bugs.python.org/issue45489 opened by bobbeyreese #45490: [meta][C API] Avoid C macro pitfalls and usage of static inlin https://bugs.python.org/issue45490 opened by vstinner Most recent 15 issues with no replies (15) ========================================== #45490: [meta][C API] Avoid C macro pitfalls and usage of static inlin https://bugs.python.org/issue45490 #45489: ForwardRef does not support | operator https://bugs.python.org/issue45489 #45488: Powershell Commands no Longer Recognised https://bugs.python.org/issue45488 #45486: Stop using internal APIs in _xxsubinterpretersmodule.c. https://bugs.python.org/issue45486 #45483: pure Python class that has been registered as a `collections.a https://bugs.python.org/issue45483 #45472: Add public C API for partial "unicode-escape" and "raw-unicode https://bugs.python.org/issue45472 #45466: Simple curl/wget-like download functionality in urllib (like h https://bugs.python.org/issue45466 #45463: Documentation inconsistency on the number of identifiers allow https://bugs.python.org/issue45463 #45457: Documentation for ssl.load_default_certs is outdated https://bugs.python.org/issue45457 #45425: There is an error in the Chinese documentation of contextlib.A https://bugs.python.org/issue45425 #45388: Use JUMP_FORWARD for all forward jumps. https://bugs.python.org/issue45388 #45387: GzipFile.write should be buffered https://bugs.python.org/issue45387 #45383: PyType_FromSpec API fails to use metaclass of bases https://bugs.python.org/issue45383 #45379: Improve errors related to frozen modules. https://bugs.python.org/issue45379 #45376: Run Windows release docs build on regular CI https://bugs.python.org/issue45376 Most recent 15 issues waiting for review (15) ============================================= #45478: Send a patch to libexpat to solve _POSIX_C_SOURCE issue. https://bugs.python.org/issue45478 #45476: [C API] Disallow using PyFloat_AS_DOUBLE() as l-value https://bugs.python.org/issue45476 #45472: Add public C API for partial "unicode-escape" and "raw-unicode https://bugs.python.org/issue45472 #45468: Add support for preloading a Python script https://bugs.python.org/issue45468 #45457: Documentation for ssl.load_default_certs is outdated https://bugs.python.org/issue45457 #45455: Fill func.__doc__ lazily https://bugs.python.org/issue45455 #45452: Support crash tolerance feature for gdbm module https://bugs.python.org/issue45452 #45450: Improve syntax error for parenthesized arguments https://bugs.python.org/issue45450 #45447: IDLE: Support syntax highlighting for .pyi stub files https://bugs.python.org/issue45447 #45446: Add a way to hide fields in dataclasses https://bugs.python.org/issue45446 #45445: Fail if an invalid -X option is provided https://bugs.python.org/issue45445 #45442: Update `Virtual Environment` tutorial https://bugs.python.org/issue45442 #45435: delete misleading faq entry about atomic operations https://bugs.python.org/issue45435 #45417: Enum creation non-linear in the number of values https://bugs.python.org/issue45417 #45410: python -m test -jN: write stderr in stdout to get messages in https://bugs.python.org/issue45410 Top 10 most discussed issues (10) ================================= #45452: Support crash tolerance feature for gdbm module https://bugs.python.org/issue45452 14 msgs #44828: Using tkinter.filedialog crashes on macOS Python 3.9.6 https://bugs.python.org/issue44828 10 msgs #45384: Accept Final as indicating ClassVar for dataclass https://bugs.python.org/issue45384 10 msgs #45447: IDLE: Support syntax highlighting for .pyi stub files https://bugs.python.org/issue45447 10 msgs #45356: Calling `help` executes @classmethod @property decorated metho https://bugs.python.org/issue45356 9 msgs #45415: Assert oparg < INSTR_OFFSET() https://bugs.python.org/issue45415 9 msgs #45476: [C API] Disallow using PyFloat_AS_DOUBLE() as l-value https://bugs.python.org/issue45476 9 msgs #45435: delete misleading faq entry about atomic operations https://bugs.python.org/issue45435 8 msgs #45410: python -m test -jN: write stderr in stdout to get messages in https://bugs.python.org/issue45410 7 msgs #35134: Add a new Include/cpython/ subdirectory for the "CPython API" https://bugs.python.org/issue35134 6 msgs Issues closed (53) ================== #20028: csv: Confusing error message when giving invalid quotechar in https://bugs.python.org/issue20028 closed by corona10 #23262: webbrowser module broken with Firefox 36+ https://bugs.python.org/issue23262 closed by serhiy.storchaka #24444: In argparse empty choices cannot be printed in the help https://bugs.python.org/issue24444 closed by lukasz.langa #27580: CSV Null Byte Error https://bugs.python.org/issue27580 closed by serhiy.storchaka #27744: Add AF_ALG (Linux Kernel crypto) to socket module https://bugs.python.org/issue27744 closed by christian.heimes #27928: Add hashlib.scrypt https://bugs.python.org/issue27928 closed by christian.heimes #29410: Moving to SipHash-1-3 https://bugs.python.org/issue29410 closed by methane #36521: Consider removing docstrings from co_consts in code objects https://bugs.python.org/issue36521 closed by methane #42253: xml.dom.minidom.rst missing standalone documentation https://bugs.python.org/issue42253 closed by serhiy.storchaka #44105: tempfile.TemporaryDirectory deleted after sleep in threads https://bugs.python.org/issue44105 closed by serhiy.storchaka #44904: Classmethod properties are erroneously "called" in multiple mo https://bugs.python.org/issue44904 closed by rhettinger #45084: urllib.parse: remove deprecated functions (splittype, to_bytes https://bugs.python.org/issue45084 closed by vstinner #45203: Improve specialization stats for LOAD_METHOD and BINARY_SUBSC https://bugs.python.org/issue45203 closed by kj #45239: email.utils.parsedate_tz raises UnboundLocalError if time has https://bugs.python.org/issue45239 closed by lukasz.langa #45351: asyncio doc: List all sockets in TCP echo server using streams https://bugs.python.org/issue45351 closed by vstinner #45353: sys.modules: dictionary changed size during iteration https://bugs.python.org/issue45353 closed by gregory.p.smith #45386: xmlrpc.client unimportable due to strfmt ValueError https://bugs.python.org/issue45386 closed by lukasz.langa #45401: logging TimedRotatingFileHandler must not rename devices like https://bugs.python.org/issue45401 closed by vinay.sajip #45404: Undefined I_* macros of stropts.h when building Python 3.10 on https://bugs.python.org/issue45404 closed by christian.heimes #45405: configure fails on macOS with non-Apple clang version 13 which https://bugs.python.org/issue45405 closed by ned.deily #45412: [C API] Remove Py_OVERFLOWED(), Py_SET_ERRNO_ON_MATH_ERROR(), https://bugs.python.org/issue45412 closed by vstinner #45416: "loop argument must agree with lock" instantiating asyncio.Con https://bugs.python.org/issue45416 closed by asvetlov #45418: types.UnionType is not subscriptable https://bugs.python.org/issue45418 closed by kj #45419: DegenerateFiles.Path mismatch to Traversable interface https://bugs.python.org/issue45419 closed by jaraco #45421: Remove dead code from html.parser https://bugs.python.org/issue45421 closed by methane #45422: Data 0 cannot be plotted by matplotlib.pyplot just because som https://bugs.python.org/issue45422 closed by serhiy.storchaka #45424: ssl.SSLError: unknown error (_ssl.c:4034) https://bugs.python.org/issue45424 closed by rahullak #45426: PANDAS INSTALLATION PIP FAILED ON WINDOWS 11 https://bugs.python.org/issue45426 closed by eric.smith #45428: py_compile fails to read filenames from stdin https://bugs.python.org/issue45428 closed by serhiy.storchaka #45430: PEP 523 no longer works https://bugs.python.org/issue45430 closed by Mark.Shannon #45432: sys.argv is processed strangely under Windows https://bugs.python.org/issue45432 closed by zach.ware #45433: libpython should not be linked with libcrypt https://bugs.python.org/issue45433 closed by vstinner #45434: [C API] Clean-up the Python.h header file https://bugs.python.org/issue45434 closed by vstinner #45437: Assignment to a list of dictionary wrong https://bugs.python.org/issue45437 closed by zach.ware #45439: [C API] Move usage of tp_vectorcall_offset from public headers https://bugs.python.org/issue45439 closed by vstinner #45440: [C API] Py_IS_INFINITY() macro doesn't work in the limited C A https://bugs.python.org/issue45440 closed by vstinner #45441: Some links in the document is changed https://bugs.python.org/issue45441 closed by serhiy.storchaka #45448: PIP package installation failure for multiple packages https://bugs.python.org/issue45448 closed by eric.smith #45453: test_embed.StdPrinterTests() crash if stdout FD is not 1 or 2 https://bugs.python.org/issue45453 closed by vstinner #45454: Unable to explicitly subclass protocol when subclass has mixin https://bugs.python.org/issue45454 closed by cmeyer #45456: operator 'pass' in 'if-else' linear expression https://bugs.python.org/issue45456 closed by steven.daprano #45458: "\W" pattern with re.ASCII flag is not equivalent to "[^a-zA-Z https://bugs.python.org/issue45458 closed by serhiy.storchaka #45461: UnicodeDecodeError: 'unicodeescape' codec can't decode byte 0x https://bugs.python.org/issue45461 closed by serhiy.storchaka #45462: Speed up re.match with pre-compiled patterns https://bugs.python.org/issue45462 closed by serhiy.storchaka #45465: logging messages are needlessly reformatted for every handler https://bugs.python.org/issue45465 closed by vinay.sajip #45467: Fix incremental decoder and stream reader in the "raw-unicode- https://bugs.python.org/issue45467 closed by serhiy.storchaka #45469: lambda issue in for-loop https://bugs.python.org/issue45469 closed by christian.heimes #45470: possible bug in datetime.timestamp() https://bugs.python.org/issue45470 closed by stef #45471: _Py_path_config.stdlib_dir should not be set in Py_SetPythonHo https://bugs.python.org/issue45471 closed by eric.snow #45474: [C API] marshal.h must not use FILE* type in the limited C API https://bugs.python.org/issue45474 closed by vstinner #45479: clean up Py_UniversalNewlineFgets https://bugs.python.org/issue45479 closed by benjamin.peterson #45481: gc is disabled without explict calling gc.disable() https://bugs.python.org/issue45481 closed by eric.smith #45482: [C API] Move _PyNamespace_New() to the internal C API https://bugs.python.org/issue45482 closed by vstinner

1 0

What is __int__ still useful for?
by Antoine Pitrou Oct. 15, 2021

Oct. 15, 2021

Hello, It used to be that defining __int__ allowed an object to be accepted as an integer from various functions, internal and third-party, thanks to being implicitly called by e.g. PyLong_AsLong. Today, and since bpo-37999, this is no longer the case. It seems that __int__ has now become a strict equivalent to __trunc__. Of course, user code can still look up and call the __int__ method explicitly (or look up the nb_int slot, in C), but that's a bit of anti-pattern. Is there a point in having three special methods __index__, __int__ and __trunc__, if two are them are practically interchangeable? Regards Antoine.

9 15

PEP-646 question: unpacking into single Generic parameter
by willi＠schinmeyer.de Oct. 14, 2021

Oct. 14, 2021

Hello, first of all, thanks for working on variadic generics! I look forward to them. My question: all the examples in https://www.python.org/dev/peps/pep-0646/ unpack into variadic arguments. But can I write code like this? ``` Ts = TypeVarTuple("Ts") def enumerate_args(f: Callable[[*Tuple[int, Ts]]], *args: *Ts): f(*enumerate(args)) ``` In particular I'm talking about the `*Tuple[int, Ts]` syntax. All the examples from the PEP use `*Ts` so I don't know if this is legal, but I hope so. This should probably be clarified in the PEP. -- Best regards, Willi Schinmeyer

3 2

Heads up: core sprint coming up!
by Łukasz Langa Oct. 13, 2021

Oct. 13, 2021

Hi there, I have a reminder for you if you are: - a core developer; or - on the triage team; or - a mentee in core-mentorship. We are running an online core sprint next week (October 18 - 24). "Online" means on our internal Core Python Discord. Join us there to plan and chat! You can even get a meal stipend from the PSF for the duration of the sprint! If you need a Discord invite, write to me directly and I'll sort you out 😎 What if you'd like to participate but you're not a core dev, triager, nor mentee? Well, this is a closed event to let our team coordinate in real time. We'd do this one in person if that was possible. In fact, this is how we have done it between 2016 and 2019. The goals don't change though: to have a small event separate from major conferences and free from distractions so we can focus on discussing key changes and making larger code changes. Instead, come to the sprints at PyCon US! And in the mean time, I suggest you pick up bugs on bugs.python.org <http://bugs.python.org/> and make pull requests. I'll review it personally if you'd like 🤓 -- Łukasz Langa CPython Developer in Residence Python Software Foundation

1 0

Re: RFC on Callable Type Syntax
by Erik Demaine Oct. 12, 2021

Oct. 12, 2021

On Mon, 11 Oct 2021, Guido van Rossum wrote: > I always found the following more obvious: > > def data_to_table(d: Iterable[Mapping[str, float]], *, sort: bool = False, reversed: bool = False) -> Table: > ... > > @dataclass > class Stream: > converter: data_to_table | None > > def add_converter(self, converter: data_to_table) -> None: > self.converter = converter Another possibility would be that functions can't be used as their types directly, but need a casting operator like so: ``` def add_converter(self, converter: typeof(data_to_table)) -> None: self.converter = converter ``` It's too bad `type` is already taken, but `typeof` is what TypeScript calls it: https://www.typescriptlang.org/docs/handbook/2/typeof-types.html -- perhaps there is a better distinguishing name, but I'll keep using `typeof` for the sake of this discussion. For static analysis, `typing.typeof` should only be used for variables that are assigned (certain) constant values and aren't re-assigned. Potentially this could be a much more general solution. Maybe interesting return values for `typeof` could be defined for `dict`s, `namedtuple`s, etc. And it could co-exist with the (P, Q) -> R proposal (which, as you probably know, is how TypeScript expresses function types). > One disadvantage of this is that now arguments HAVE TO be named which > raises questions: > - should they be considered at type checking time? > - how to express "I don't care"? I think it'd be natural for types constructed via `typeof` to include the argument names; I'd intend for an object "just like" this one. But you could imagine another type operator that drops details like this, e.g. `typing.forget_argument_names(typing.typeof(data_table))`. Erik -- Erik Demaine | edemaine(a)mit.edu | http://erikdemaine.org/

3 3

Why doesn't peephole optimise away operations with fast locals?
by Patrick Reader Oct. 11, 2021

Oct. 11, 2021

Consider sequences of bytecode operations on fast locals like the following: >>> def f(x): # declare as parameter so that x is a fast local ... x ... >>> dis(f) 2 0 LOAD_FAST 0 (x) 2 POP_TOP 4 LOAD_CONST 0 (None) 6 RETURN_VALUE Given the following assumptions: - a LOAD_FAST cannot possibly have any side-effects outside the interpreter stack [1] - a POP_TOP immediately undoes LOAD_FAST's effects on the interpreter stack I am wondering: why doesn't the peephole optimiser remove these opcode constructs? Is it maybe because of PEP 626 - an introspection tool needs to know that the variable is being used there? Although surely in that case the peephole optimiser could just replace it with a single NOP? (c.f.: https://bugs.python.org/issue42719) Or is one of my assumptions wrong? [1]: global variables are probably supposed to have the same guarantee, but in practice this is not the case, which is why I'm focusing on the _FAST ones; see for example https://ato.pxeger.com/run?1=m72soLIkIz9vwYKlpSVpuhY3JyXnJBYXK_hWumQml2ikAA…

6 11

Summary of Python tracker Issues
by Python tracker Oct. 8, 2021

Oct. 8, 2021

ACTIVITY SUMMARY (2021-10-01 - 2021-10-08) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 7374 (+33) closed 49772 (+38) total 57146 (+71) Open issues with patches: 2920 Issues opened (51) ================== #33125: Windows 10 ARM64 platform support https://bugs.python.org/issue33125 reopened by steve.dower #45163: Haiku build fix https://bugs.python.org/issue45163 reopened by ned.deily #45344: Have zipapp respect SOURCE_DATE_EPOCH https://bugs.python.org/issue45344 opened by bign8 #45347: datetime subject to rounding? https://bugs.python.org/issue45347 opened by piro #45351: asyncio doc: List all sockets in TCP echo server using streams https://bugs.python.org/issue45351 opened by olafvdspek #45352: Move documentation for typed generic forms of standard collect https://bugs.python.org/issue45352 opened by pxeger #45353: sys.modules: dictionary changed size during iteration https://bugs.python.org/issue45353 opened by idan57 #45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of https://bugs.python.org/issue45355 opened by serhiy.storchaka #45356: Calling `help` executes @classmethod @property decorated metho https://bugs.python.org/issue45356 opened by randolf.scholz #45357: Idle does not check user config for extention configuration https://bugs.python.org/issue45357 opened by CoolCat467 #45358: Bogus cookie generated after invalid cookie attribute is input https://bugs.python.org/issue45358 opened by greob #45359: TopologicalSorter is not Generic at runtime (but is in typeshe https://bugs.python.org/issue45359 opened by JacobHayes #45361: Provide a more convenient way to set an exception as "active", https://bugs.python.org/issue45361 opened by njs #45363: Use instruction offsets in co_lnotab https://bugs.python.org/issue45363 opened by serhiy.storchaka #45364: Add more documentation for ModuleSpec.loader_state. https://bugs.python.org/issue45364 opened by eric.snow #45365: concurrent.futures.Future should be suitable for use outside o https://bugs.python.org/issue45365 opened by xmorel #45366: dataclass init=False field with default works but default_fact https://bugs.python.org/issue45366 opened by simple_coder878 #45367: Specialize BINARY_MULTIPLY https://bugs.python.org/issue45367 opened by Dennis Sweeney #45369: Remove LibreSSL workarounds https://bugs.python.org/issue45369 opened by ramikg #45371: distutil's runtime_library_dir (rpath) option doesn't work wit https://bugs.python.org/issue45371 opened by christian.heimes #45373: ./configure --enable-optimizations should enable LTO https://bugs.python.org/issue45373 opened by vstinner #45374: sqlite3: Add configure option to set or auto-detect rpath to s https://bugs.python.org/issue45374 opened by chaz6 #45376: Run Windows release docs build on regular CI https://bugs.python.org/issue45376 opened by steve.dower #45378: Can't find "map" with search on docs.python.org https://bugs.python.org/issue45378 opened by gvanrossum #45379: Improve errors related to frozen modules. https://bugs.python.org/issue45379 opened by eric.snow #45380: help() appears confused about the module of typing.Annotated https://bugs.python.org/issue45380 opened by AlexWaygood #45381: IDLE: "restart shell" while receiving output disables program https://bugs.python.org/issue45381 opened by capsicumw #45382: platform() is not able to detect windows 11 https://bugs.python.org/issue45382 opened by sahsariga111 #45383: PyType_FromSpec API fails to use metaclass of bases https://bugs.python.org/issue45383 opened by seberg #45384: Accept Final as indicating ClassVar for dataclass https://bugs.python.org/issue45384 opened by GBeauregard #45386: xmlrpc.client unimportable due to strfmt ValueError https://bugs.python.org/issue45386 opened by rtobar2 #45387: GzipFile.write should be buffered https://bugs.python.org/issue45387 opened by rhpvorderman #45388: Use JUMP_FORWARD for all forward jumps. https://bugs.python.org/issue45388 opened by Mark.Shannon #45389: https://docs.python.org/3/objects.inv still points to 3.9 https://bugs.python.org/issue45389 opened by gaborjbernat #45390: asyncio.Task doesn't propagate CancelledError() exception corr https://bugs.python.org/issue45390 opened by pagliaricci.m #45391: 3.10 objects.inv classifies UnionType as data https://bugs.python.org/issue45391 opened by gaborjbernat #45392: docstring of "type" could use an update https://bugs.python.org/issue45392 opened by mark.dickinson #45393: help() on operator precedence has confusing entries "await" "x https://bugs.python.org/issue45393 opened by MFH #45395: Frozen stdlib modules are discarded if custom frozen modules a https://bugs.python.org/issue45395 opened by eric.snow #45396: Custom frozen modules get ignored. https://bugs.python.org/issue45396 opened by eric.snow #45397: Doc for turtle.write missing the tuple part of the font param https://bugs.python.org/issue45397 opened by jggammon #45399: Remove hostflags from PySSLContext https://bugs.python.org/issue45399 opened by ramikg #45401: logging TimedRotatingFileHandler must not rename devices like https://bugs.python.org/issue45401 opened by vstinner #45404: Undefined I_* macros of stropts.h when building Python 3.10 on https://bugs.python.org/issue45404 opened by etale-cohomology #45405: Python does not configure on darwin using public llvm / clang https://bugs.python.org/issue45405 opened by debohman #45406: inspect.getouterframes() tracebacks when $CWD does not exists https://bugs.python.org/issue45406 opened by msuchy(a)redhat.com #45409: Support non-standard executables in venv https://bugs.python.org/issue45409 opened by steve.dower #45410: python -m test -jN: write stderr in stdout to get messages in https://bugs.python.org/issue45410 opened by vstinner #45411: Add Mimetypes for Subtitle Files https://bugs.python.org/issue45411 opened by Josephine-Marie #45412: [C API] Remove Py_OVERFLOWED(), Py_SET_ERRNO_ON_MATH_ERROR(), https://bugs.python.org/issue45412 opened by vstinner #45413: Add install scheme for virtual environments https://bugs.python.org/issue45413 opened by FFY00 Most recent 15 issues with no replies (15) ========================================== #45399: Remove hostflags from PySSLContext https://bugs.python.org/issue45399 #45388: Use JUMP_FORWARD for all forward jumps. https://bugs.python.org/issue45388 #45387: GzipFile.write should be buffered https://bugs.python.org/issue45387 #45386: xmlrpc.client unimportable due to strfmt ValueError https://bugs.python.org/issue45386 #45384: Accept Final as indicating ClassVar for dataclass https://bugs.python.org/issue45384 #45383: PyType_FromSpec API fails to use metaclass of bases https://bugs.python.org/issue45383 #45380: help() appears confused about the module of typing.Annotated https://bugs.python.org/issue45380 #45379: Improve errors related to frozen modules. https://bugs.python.org/issue45379 #45376: Run Windows release docs build on regular CI https://bugs.python.org/issue45376 #45374: sqlite3: Add configure option to set or auto-detect rpath to s https://bugs.python.org/issue45374 #45369: Remove LibreSSL workarounds https://bugs.python.org/issue45369 #45365: concurrent.futures.Future should be suitable for use outside o https://bugs.python.org/issue45365 #45364: Add more documentation for ModuleSpec.loader_state. https://bugs.python.org/issue45364 #45363: Use instruction offsets in co_lnotab https://bugs.python.org/issue45363 #45359: TopologicalSorter is not Generic at runtime (but is in typeshe https://bugs.python.org/issue45359 Most recent 15 issues waiting for review (15) ============================================= #45412: [C API] Remove Py_OVERFLOWED(), Py_SET_ERRNO_ON_MATH_ERROR(), https://bugs.python.org/issue45412 #45410: python -m test -jN: write stderr in stdout to get messages in https://bugs.python.org/issue45410 #45409: Support non-standard executables in venv https://bugs.python.org/issue45409 #45401: logging TimedRotatingFileHandler must not rename devices like https://bugs.python.org/issue45401 #45396: Custom frozen modules get ignored. https://bugs.python.org/issue45396 #45395: Frozen stdlib modules are discarded if custom frozen modules a https://bugs.python.org/issue45395 #45391: 3.10 objects.inv classifies UnionType as data https://bugs.python.org/issue45391 #45388: Use JUMP_FORWARD for all forward jumps. https://bugs.python.org/issue45388 #45386: xmlrpc.client unimportable due to strfmt ValueError https://bugs.python.org/issue45386 #45371: distutil's runtime_library_dir (rpath) option doesn't work wit https://bugs.python.org/issue45371 #45369: Remove LibreSSL workarounds https://bugs.python.org/issue45369 #45367: Specialize BINARY_MULTIPLY https://bugs.python.org/issue45367 #45363: Use instruction offsets in co_lnotab https://bugs.python.org/issue45363 #45359: TopologicalSorter is not Generic at runtime (but is in typeshe https://bugs.python.org/issue45359 #45358: Bogus cookie generated after invalid cookie attribute is input https://bugs.python.org/issue45358 Top 10 most discussed issues (10) ================================= #29410: Moving to SipHash-1-3 https://bugs.python.org/issue29410 23 msgs #45382: platform() is not able to detect windows 11 https://bugs.python.org/issue45382 15 msgs #36521: Consider removing docstrings from co_consts in code objects https://bugs.python.org/issue36521 11 msgs #45335: Default TIMESTAMP converter in sqlite3 ignores UTC offset https://bugs.python.org/issue45335 8 msgs #45371: distutil's runtime_library_dir (rpath) option doesn't work wit https://bugs.python.org/issue45371 7 msgs #45391: 3.10 objects.inv classifies UnionType as data https://bugs.python.org/issue45391 7 msgs #45020: Freeze all modules imported during startup. https://bugs.python.org/issue45020 6 msgs #45116: Performance regression 3.10b1 and later on Windows: Py_DECREF( https://bugs.python.org/issue45116 5 msgs #45262: crash if asyncio is used before and after re-initialization if https://bugs.python.org/issue45262 5 msgs #45339: concurrent.future.ThreadPoolExecutor should parameterize class https://bugs.python.org/issue45339 5 msgs Issues closed (39) ================== #34804: Repetition of 'for example' in documentation https://bugs.python.org/issue34804 closed by lukasz.langa #35970: no help flag in base64 util https://bugs.python.org/issue35970 closed by ammar2 #40321: urllib.request does not support HTTP response status code 308 https://bugs.python.org/issue40321 closed by lukasz.langa #40418: Small Refactoring: Use bytes.hex() in secrets.token_hex() https://bugs.python.org/issue40418 closed by Dennis Sweeney #41197: Async magic methods in contextlib.closing https://bugs.python.org/issue41197 closed by njs #44050: [subinterpreters] _PyImport_FixupExtensionObject() regression https://bugs.python.org/issue44050 closed by lukasz.langa #44687: io.BufferedReader:peek() closes underlying file, breaking peek https://bugs.python.org/issue44687 closed by steve.dower #44998: macOS build test failure https://bugs.python.org/issue44998 closed by ned.deily #45119: test_signal.test_itimer_virtual() failed on AMD64 Fedora Rawhi https://bugs.python.org/issue45119 closed by vstinner #45288: Inspect - Added sort_result parameter on getmembers function. https://bugs.python.org/issue45288 closed by terry.reedy #45302: 10 built-in functions need non-None .__text_signature__ https://bugs.python.org/issue45302 closed by rhettinger #45324: The frozen importer should capture info in find_spec(). https://bugs.python.org/issue45324 closed by eric.snow #45328: http.client.HTTPConnection doesn't work without TCP_NODELAY https://bugs.python.org/issue45328 closed by lukasz.langa #45329: pyexpat: segmentation fault when `--with-system-expat` is spec https://bugs.python.org/issue45329 closed by serhiy.storchaka #45337: Create venv with pip fails when target dir is under userappdat https://bugs.python.org/issue45337 closed by steve.dower #45338: Add key argument to collections.Counter https://bugs.python.org/issue45338 closed by kubataytekin #45341: Update "Python Package Index" in Docs https://bugs.python.org/issue45341 closed by StevenHsuYL #45343: Update bundled pip to 21.2.4 and setuptools to 58.1.0 https://bugs.python.org/issue45343 closed by lukasz.langa #45345: Equalized lists depend to each other after equalizing https://bugs.python.org/issue45345 closed by eric.smith #45346: Some "truthy" crept in https://bugs.python.org/issue45346 closed by rhettinger #45348: math.log(243, 3) value issue https://bugs.python.org/issue45348 closed by tim.peters #45349: configparser.ConfigParser: 2 newlines at end of file (EOF) https://bugs.python.org/issue45349 closed by martin.panter #45350: configure incorrectly ignores pkg-config information for libff https://bugs.python.org/issue45350 closed by pablogsal #45354: test_winconsoleio fails on Windows 11 https://bugs.python.org/issue45354 closed by steve.dower #45360: xml.etree.ElementTree: add feature in write to create director https://bugs.python.org/issue45360 closed by serhiy.storchaka #45362: dis does not work with the new optimized ops https://bugs.python.org/issue45362 closed by kj #45368: ~(True) and ~(False) gives incorrect result https://bugs.python.org/issue45368 closed by serhiy.storchaka #45370: Typo in pep-0636 page https://bugs.python.org/issue45370 closed by xtreak #45372: Unwarranted "certificate has expired" when urlopen-ing R3 site https://bugs.python.org/issue45372 closed by aivarannamaa #45375: Windows assertion in out-of-tree debug build https://bugs.python.org/issue45375 closed by steve.dower #45377: Default python 3 docs still pointing to 3.9.7 https://bugs.python.org/issue45377 closed by eric.smith #45385: Fix reference leak from descr_check https://bugs.python.org/issue45385 closed by corona10 #45394: pip install numpy not working in 3.11.0a on macOS 11.6 https://bugs.python.org/issue45394 closed by christian.heimes #45398: Bugs in set operations in 3.8.10 (detected in Win7) https://bugs.python.org/issue45398 closed by bfx683 #45400: test_name_error_suggestions_do_not_trigger_for_too_many_locals https://bugs.python.org/issue45400 closed by vstinner #45402: ERROR: test_sundry (test.test_tools.test_sundry.TestSundryScri https://bugs.python.org/issue45402 closed by vstinner #45403: test_sys: test_stdlib_dir() fails when Python is built outside https://bugs.python.org/issue45403 closed by vstinner #45407: Support buffer protocol for _struct.Struct format argument https://bugs.python.org/issue45407 closed by corona10 #45408: [fuzzer] Weird input with continuation and newlines causes nul https://bugs.python.org/issue45408 closed by pablogsal

1 0

We should accept Final as indicating ClassVar for dataclasses
by Gregory Beauregard Oct. 8, 2021

Oct. 8, 2021

(copied from typing on advice since this affects runtime behavior) PEP 591 for Final states "Type checkers should infer a final attribute that is initialized in a class body as being a class variable. Variables should not be annotated with both ClassVar and Final." ClassVar is used to indicate library behavior in dataclasses. Therefore, I propose accepting the Final attribute as an indicator of a ClassVar in dataclass class bodies in order to be better compatible with the Final PEP. One edge case that would need to be handled is when someone might explicitly mark a dataclass field Final: a: Final[int] = dataclasses.field(init=False, default=10) I've opened an issue as well and would greatly appreciate any feedback: https://bugs.python.org/issue45384 I appreciate your time, Gregory Beauregard

1 0

CPython is participating in hacktoberfest (with caveat)
by Mariatta Oct. 8, 2021

Oct. 8, 2021

Hello, I'm happy to announce that CPython project is participating in hacktoberfest. I have just added the hacktoberfest topic to CPython's GitHub Repo ( https://github.com/python/cpython). Caveat: if we end up receiving too many spammy, invalid, and low-quality PRs which just adds burden to maintainers, we will opt-out of this immediately. I will be watching incoming PRs for the next week to determine whether we should opt-out. If you're participating in hacktoberfest, your contribution to the repo will count. Instead of receiving a t-shirt, you can also choose to have a tree planted on your behalf, which I personally recommend. If you're a core dev, you can also participate and get rewarded by reviewing and merging pull requests. If you spot spammy PRs, please add either the "spam" or "invalid" label. Another way to participate is by donating to participating open source projects. You can donate to CPython development through GitHub Sponsors ( https://github.com/sponsors/python) Hacktoberfest <https://hacktoberfest.digitalocean.com/> is an annual event organized by Digital Ocean, now in the 8th year, where aspiring contributors are encouraged to contribute to open source projects. I'm a member of the Hacktoberfest Advisory council <https://hacktoberfest.digitalocean.com/advisory-council>. If you have concerns, feedback, and suggestions about this event, please do let me know and I'll be sure to pass it along. Thank you.

1 0