Mailman 3 February 2021 - Python-Dev

Move support of legacy platforms/architectures outside Python
by Victor Stinner 24 Feb '21

24 Feb '21

Hi, I propose to actively remove support for *legacy* platforms and architectures which are not supported by Python according to PEP 11 rules: hardware no longer sold and end-of-life operating systems. The removal should be discussed on a case by case basis, but I would like to get an agreement on the overall idea first. Hobbyists wanting to support these platforms/archs can continue to support them with patches maintained outside Python. For example, I consider that the 16-bit m68k architecture is legacy, whereas the OpenBSD platform is still actively maintained. I already know that there will be a strike back: "oh no, you must continue to support my architecture" and "their existing code should stay and doesn't cost anything to maintain". Python is maintained by volunteers, the majority is contributing in their free time, so people are free to use their free time as they want. You cannot ask core developers to support your favorite *legacy* platform/architecture if they don't want to. In short, I propose to move maintenance of the legacy platforms/archs outside Python: people are free to continue support them as patches. -- Concrete example: Christian Heimes proposed to drop support for 31-bit s390 Linux: https://bugs.python.org/issue43179 The lack of clear definition on how a platform is supported or not confuses users who consider that their favorite platform/arch is supported, whereas core developers don't want to support it since it would be too much work. In fact, the PEP 11 has clear and explicit rules: https://www.python.org/dev/peps/pep-0011/#supporting-platforms A platform is only considered as supported if the following two conditions are met: 1) a core developer needs to volunteer to maintain platform-specific code 2) a stable buildbot must be provided Last October, I proposed to drop Solaris support (bpo-42173). Jakub Kulik stepped in and proposed some Solaris patches, so I abandoned my idea. But I still don't see any running Solaris buildbot worker, and there is still no core developer volunteer to maintain Solaris support. It's unclear to me if Python has "best effort" support for Solaris, of if Solaris is "not supported". -- Over the years, Python was ported to tons of platforms and CPU architectures. It didn't matter if the platform or the architecture was commonly used or not. 30 years later, Python still has the code for many legacy platforms and architectures. Some hardware is no longer sold but kept alive by hobbyists, especially members of retro computing groups. Some Linux distributions like Gentoo and Debian are trying to support most architectures which are supported by these hobbyist groups, whereas some other distributions like Ubuntu are limited to a few platforms. For example, Ubuntu 20.4.2 LTS server supports 4 architectures: x86-64, AArch64 (ARM), POWER and s390x. I guess that the difference between Debian and Ubuntu is that Ubuntu is a Canonical product, Canonical sells professional support and so cannot support too many architectures. Each architecture support requires to build all packages on it, tests the packages, have experts who fix issues specific to this arch, etc. -- Python has different kinds of platform and architecture supports. In practice, I would say that we have: * (1) Fully supported. Platform/architecture used by core developers and have at least one working buildbot worker: fully supported. Since core developers want to use Python on their machine, they fix issues as soon as they notice them. Examples: x86-64 on Linux, Windows and macOS. * (2) Best effort. Platform/architecture which has a buildbot worker usually not used by core developers. Regressions (buildbot failures) are reported to bugs.python.org, if someone investigates and provides a fix, the fix is merged. But there is usually no "proactive" work to ensure that Python works "perfectly" on these platforms. Example: FreeBSD/x86-64. * (3) Not (officially) supported. We enter the blurry grey area. There is no buildbot worker, no core dev use it, but Python contains code specific to these platforms/architectures. Example: 16-bit m68k and 31-bit s390 architectures, OpenBSD. The Rust programming language has 3 categories of Platform Support, the last one is : "Tier 3 platforms are those which the Rust codebase has support for, but which are not built or tested automatically, and may not work. Official builds are not available." https://doc.rust-lang.org/nightly/rustc/platform-support.html Rust Tier 3 looks like our "Not supported" category. Maybe we should have a better definition in the PEP 11 of our 3 support levels. There is also a 4th category: platforms/archs which are really not supported, like they legacy ones for which we removed the code :-) Examples: BeOS, MacOS 9, platforms with no thread support, etc. -- There is also a "Stable Buildbot" category used by the "Release Status" web page: https://buildbot.python.org/all/#/release_status There is no clear rule why is a worker is added to that list or not. For example, there are two FreeBSD workers which runs FreeBSD CURRENT: the FreeBSD development branch. I don't think that it should be declared as "stable". Victor -- Night gathers, and now my watch begins. It shall not end until my death.

17 40

PEP 653: Precise Semantics for Pattern Matching
by Mark Shannon 23 Feb '21

23 Feb '21

Hi everyone, I'd like to announce a new PEP. https://www.python.org/dev/peps/pep-0653/ It builds on PEP 634 (Structural Pattern Matching: Specification), adding: More precise semantics. A bit more customization for complex classes. Its also a bit more robust and should be faster (eventually). The syntax is unchanged, and in most cases the semantics are the same. As always, comments and suggestions are welcome. Cheers, Mark.

5 15

Security releases of CPython
by Michał Górny 22 Feb '21

22 Feb '21

Hello, I'm the primary maintainer of CPython packages in Gentoo. I would like to discuss possible improvement to the release process in order to accelerate releasing security fixes to users. I feel that vulnerability fixes do not make it to end users fast enough. For example, according to the current release schedules for 3.9 and 3.8, the bugfix releases are planned two months apart. While the release is expected to happen in the next few days, both versions are known to be vulnerable for almost a month! Ironically, things look even worse for security-supported versions. Please correct me if I'm wrong but both 3.7 and 3.6 seem to be behind schedule (planned for Jan 15th), and they are known to be vulnerable since mid-October. In my opinion, this causes three issues: 1. Users using official releases are exposed to security vulnerabilities for prolonged periods of time. 2. When releases happen, security fixes are often combined with many other changes. This causes problems for distribution maintainers who, on one hand, would like to deploy the security fixes to production versions ASAP, and on the other, would prefer that the new version remained in testing for some time due to the other changes. 3. Effectively, it means that distribution developers need to track and backport security fixes themselves. In the end, this means a lot of duplicate work. I think that security fixes are important enough to justify not sticking to a strict release schedule. Therefore, I would like to propose that if vulnerability fixes are committed, new releases are made as frequently as necessary and as soon as possible (i.e. possibly giving some time for testing) rather than according to a strict schedule. Furthermore, I think that at least for branches that are in higher level of maintenance than security, it could make sense to actually make security releases (e.g. 3.9.1.x) that would include only security fixes without other changes. What do you think? -- Best regards, Michał Górny

13 19

Inadequate error reporting during function call setup stage
by Paul Sokolovsky 22 Feb '21

22 Feb '21

Hello, Here's example: Traceback (most recent call last): File "pseudoc_tool.py", line 91, in <module> first_class_function_value(func, **pass_params) TypeError: print() got an unexpected keyword argument 'noann' Ok, which "print" do you mean, dear CPython? I have a dozen of print functions (mostly methods) around. So I now need to reduce to printf debugging to find which one is actually meant. And what if that happens in 3rd-party library installed in /usr/lib? On a production system? The most interesting fact is that despite functions has always been first-class values and 30-years language history, such issues are still there. I'm fixing this right away in my Pycopy dialect, because while it's minimalist and I tremble over each half-byte added, but it's Python first of all, and Python means good error reporting. Well, actually I don't fix it right away, but work around, put putting source loc in TypeError's error message. And here's an idea of what the real fix should be: The fact that CPython (and Pycopy) sets up function's frame "outside" the function execution is an implementation detail. The model of execution should be: there's a CALL_FUNCTION/CALL_FUNCTION_KW opcode, and after it executes, the context switches to the target function, and that's *when* argument checking happens. That would give the desired behavior that the last entry in traceback is the actual function which was called and which failed parameter checks. Again, it's an implementation detail if some project doesn't do it like that, and performs param checking before actually pointing VM at the new bytecode function (from which location traceback will be automatically constructed). If it does that, it just still need to append an additional traceback entry pointing to the beginning of the target function. I now implemented that too, and now everything makes sense: Traceback (most recent call last): File "pseudoc_tool.py", line 91, in <module> File "../xforms.py", line 25, in print TypeError: unexpected keyword argument 'noann' -- Best regards, Paul mailto:pmiscml@gmail.com

4 5

Deprecate support for mingw - add to PEP 11
by ucodery＠gmail.com 22 Feb '21

22 Feb '21

I think perhaps we should admit that this build system is no longer supported. From everything I can tell, the mingw project is no longer maintained. The project's site, mingw.org, is no longer live; the project on sourceforge, although still downloaded daily, had its last commit almost 3 years ago - a commit which changed the official project URI to a new link that now is also dead. Looking over BPO there are a little over 50 bugs open against mingw, but only 7 that have any meaningful activity within the last three years. Three of those issues explicitly mention mingw-w64 which is an active fork of the original mingw project (active homepage, commits almost daily, new release within the last 6 months) and I wonder if this is the project the other 4 projects meant by "mingw"? Ideally any features and flags in the code base for mingw would be checked to already be working with mingw-w64 or else modified to work, but this would require a sponsor for this platform, which appears to be missing. Further, there is no buildbot for mingw, which should be a requirement to be a fully supported platform, as per this PEP. This potential work appears non-trivial with a cursory look at the mingw-w64-python pacman project, which contains close to 100 patch files. I am purposing instead that mingw be deprecated and, if a sponsor comes along, mingw-w64 can become re-supported, or newly supported depending on you point of view, as allowed by the PEP.

5 5

minor PR 22981 waiting review ~20 days: https://github.com/python/cpython/pull/22981
by Alfred Perlstein 22 Feb '21

22 Feb '21

PR 22981 is a minor update to doctest to allow it to safely consume modules which contain objects that cause inspect.unwrap to throw. I believe the review comments in the PR have been addressed at this point for ~20 days. The patch is relatively small, reviews are done, but it is unmerged. Is there a way to "bump" the request for review? thank you, -Alfred

2 2

PEP 597 bikeshedding: envvar / option name.
by Inada Naoki 21 Feb '21

21 Feb '21

I am updating PEP 597 to include discussions in the thread. Before finishing the PEP, I need to decide the option name. I used PYTHONWARNDEFAULTENCODING (envvar name) / warn_default_encoding (-X option and sys.flags name) before, but it seems too long and hard to type, easy to misspell. Currently, I use PYTHONWARNENCODING / warn_encoding, but it is not so explicit. Which name is the best balance between explicitness and readability? * PYTHONWARNENCODING / warn_ecoding * PYTHONWARNOMITENCODING / warn_omit_encoding * PYTHONWARNDEFAULTENCODING / warn_default_encoding * Anything else Regards, -- Inada Naoki <songofacandy(a)gmail.com>

4 7

Happy 30th Birthday, Python!
by Larry Hastings 21 Feb '21

21 Feb '21

I guess we forgot to observe it yesterday, but: February 19, 1991, was the day Guido first posted Python 0.9.1 to alt.sources: https://groups.google.com/g/alt.sources/c/O2ZSq7DiOwM/m/gcJTvCA27lMJ Happy 30th birthday, Python! //arry/

2 1

Anyone interested in leading a sprint at PyCascades tomorrow?
by Brett Cannon 20 Feb '21

20 Feb '21

It's short notice, but PyCascades is hosting sprints on Sunday, Feb. 21 from 09:00 - 12:30 PST tomorrow and the organizers wanted to see if anyone happened to be up for leading or helping out anyone who wants to sprint on CPython. https://2021.pycascades.com/program/sprints/

1 0

Clarification on the removal of generator-based coroutines in 3.10
by Luciano Ramalho 19 Feb '21

19 Feb '21

<DISCLAIMER>I am not a Python core developer, but my question relates to changes that are expected in Python 3.10, so I felt this was the best forum to ask. Please let me know if I should discuss this elsewhere or file a documentation bug.</DISCLAIMER> First of all, thank you Yuri Selivanov and everybody who contributed to the overhaul of the asyncio docs which are now much better organized. In the process of updating "Fluent Python" to cover Python 3.9, I learned that the "Generator-based coroutines" section in the asyncio docs warns that "Support for generator-based coroutines is deprecated and is scheduled for removal in Python 3.10." [1] [1] https://docs.python.org/3/library/asyncio-task.html#generator-based-corouti… I don't understand that warning, given that PEP 492 lists as one of the awaitable types: * A generator-based coroutine object returned from a function decorated with types.coroutine(). Perhaps what is going to be removed is support for old style generator-based coroutines decorated with @asyncio.coroutine, or not decorated (which historically also worked)? I'd appreciate some clarification on exactly what is going to be removed. Maybe we also need updates to the Glossary [2] to bridge the gap between our previous use of the word "coroutine" (as in PEP 342) and the way we are using it today, when we are not being explicit about "native coroutines". In particular, the Glossary has no entry at all for "generator-based coroutine"—those marked with @types.coroutine. [2] https://docs.python.org/3/glossary.html Thank you for your time and help! Cheers, Luciano -- Luciano Ramalho | Author of Fluent Python (O'Reilly, 2015) | http://shop.oreilly.com/product/0636920032519.do | Technical Principal at ThoughtWorks | Twitter: @ramalhoorg

5 13