Partial support of a platform
Hi, I see more and more proposed changes to fix some parts of the code to "partially" support a platform. I remember that 5 years ago, the CPython code was "full" of #ifdef and other conditional code to support various platforms, and I was happy when we succeeded to remove support for all these old platforms like OS/2, DOS or VMS. The PEP 11 has a nice description to get a *full* support of a new platform: https://www.python.org/dev/peps/pep-0011/ But the question here is more about "partial" support. While changes are usually short, I dislike applying them to Python 2.7 and/or Python 3.6, until a platform is fully support. I prefer to first see a platform fully supported to see how much changes are required and to make sure that we get someone involved to maintain the code (handle new issues). Example of platforms: MinGW, Cygwin, OpenBSD, NetBSD, xWorks RTOS, etc. But the way, is there an exhaustive list of platforms "officially" supported by CPython? Victor
The PEP 11 has a nice description to get a *full* support of a new platform: https://www.python.org/dev/peps/pep-0011/
PEP 11 defines the endpoint, full support, and several requirements to call a platform fully supported. It would be nice if a process was defined for getting from "no support" to "full support." I think that to be as supportive as possible (no pun intended), it would make sense to grant people check-in privileges to a branch (if that's possible or desirable), or, if not, list forks which are intended to add support for various platforms which are moving toward full support. I don't know if PEP 11 is the right place to track such activity, relatively few updates should be required. I doubt it will be an onerous burden. Something like: * ButterflyOS - Victor Stinner (victor.stinner@gmail.com) is working to add CPython support for this platform on this Git branch: https://github.com/python/Butterfly * MothOS - Skip Montanaro (skip.montanaro@gmail.com) is working to add CPython support for this platform on this Git branch: https://github.com/smontanaro/Moth Interested parties would be directed to contact the pilots of those branches if they wanted to contribute. Skip
On 06Nov2017 0941, Victor Stinner wrote:
[SNIP]
But the question here is more about "partial" support.
While changes are usually short, I dislike applying them to Python 2.7 and/or Python 3.6, until a platform is fully support. I prefer to first see a platform fully supported to see how much changes are required and to make sure that we get someone involved to maintain the code (handle new issues).
Example of platforms: MinGW, Cygwin, OpenBSD, NetBSD, xWorks RTOS, etc.
I appreciate the desire for changes to be made upstream, especially on code that changes frequently enough to make it difficult to patch reliably (this is basically the entire premise behind my PEP 551). At the same time, I don't like carrying the burden of code for platforms we do not support (hence PEP 551 doesn't really add any interesting code). There is a balance to be found here, though. I don't believe CPython *partially* supports any platforms - either they are fully supported or they are not supported. However, we can and should do things that help other people support their platforms, such as making sure build options can be specified by environment variables. At the same time, we can and should *exclude* things that require platform-specific testing in core (for example, predefined options for building for a specific platform). Similarly, if someone wanted to add specific platform support to a stdlib module via "if sys.platform ...", I'd be hesitant. However, if they refactored it such that it was easier *for a custom build of CPython* to provide/omit certain features (e.g. https://github.com/python/cpython/blob/30f4fa456ef626ad7a92759f492ec7a268f7a... ) then I'd be more inclined to accept it - but only if there was no behavioural change on supported platforms. Does that make sense? I'm not sure whether we ought to capture some general guidelines somewhere on how to make decisions around this, but we should certainly have the discussion here first anyway. Cheers, Steve
2017-11-06 22:16 GMT+01:00 Steve Dower
I don't believe CPython *partially* supports any platforms - either they are fully supported or they are not supported.
Ok. So there are two questions: * Where is the list of platforms "endorsed" by CPython ("fully supported") * How can we decide when to start to support support a new platform? Usually, developers begin with a nice looking change: short, not intrusive. But later, the real change comes, and it's much larger and uglier :-) And the rationale for the first change is "come on, it's just a few lines!". And I have troubles to justify to reject such patch. Slowly, I'm pointing people to the right section of the PEP 11. Victor
2017-11-06 22:24 GMT+01:00 Antoine Pitrou
We support POSIX-compatible platforms. Do OpenBSD and NetBSD need special care?
Aha, "POSIX", you are funny Antoine :-D If it was a single #ifdef in the whole code base, I wouldn't have to start such thread on python-dev :-) Open issues with "OpenBSD" in the title: 31630: math.tan has poor accuracy near pi/2 on OpenBSD and NetBSD 31631: test_c_locale_coercion fails on OpenBSD 31635: test_strptime failure on OpenBSD 31636: test_locale failure on OpenBSD 25342: test_json segfault on OpenBSD 25191: test_getsetlocale_issue1813 failed on OpenBSD 23470: OpenBSD buildbot uses wrong stdlib 12588: test_capi.test_subinterps() failed on OpenBSD (powerpc) 12589: test_long.test_nan_inf() failed on OpenBSD (powerpc) CPython already contais 11 "#ifdef (...) __OpenBSD__" in C and 11 sys.platform.startswith('openbsd') in Python. Supporting a "POSIX" platform requires some changes :-) Open issues with "NetBSD" in the title: 31925: [NetBSD] test_socket creates too many locks 30512: CAN Socket support for NetBSD 31927: Fix compiling the socket module on NetBSD 8 and other issues 31630: math.tan has poor accuracy near pi/2 on OpenBSD and NetBSD 31894: test_timestamp_naive failed on NetBSD CPython already contains 17 "#ifdef (...) __NetBSD__" and 7 "platform.startswith('netbsd')" in Python. MinGW: ... hum, there are 57 open MinGW issues, use the bug tracker to list them ;-) Cygwin: ... hum, there are 23 open Cygwin issues, use the bug tracker to list them ;-) etc. Victor
On Mon, 6 Nov 2017 22:49:06 +0100
Victor Stinner
CPython already contais 11 "#ifdef (...) __OpenBSD__" in C and 11 sys.platform.startswith('openbsd') in Python. Supporting a "POSIX" platform requires some changes :-)
Yes... So, the question is: does OpenBSD already maintain a Python port? If so, then we should have them contribute their patches instead of fixing issues ourselves. If they don't want too, then we have no obligation to maintain them. OTOH, if you and Serhiy want to maintain them, you can do so. But we shouldn't promise anything for the future. The reason that testing on them is interesting, IMHO, is to chase potential Linux-isms in our code base. Circumventing {Free,Open,Net}BSD-specific deficiences is not. To put things differently: if a commit breaks {Free,Open,Net}BSD but leaves other platforms intact, I don't think it should be reverted. Regards Antoine.
2017-11-06 23:07 GMT+01:00 Antoine Pitrou
The reason that testing on them is interesting, IMHO, is to chase potential Linux-isms in our code base. Circumventing {Free,Open,Net}BSD-specific deficiences is not.
Serhiy found at least an interesting issue thanks to OpenBSD, a bug in memory debug hooks: https://bugs.python.org/issue31626 Victor
06.11.17 23:24, Antoine Pitrou пише:
On Mon, 6 Nov 2017 18:41:30 +0100 Victor Stinner
wrote: Example of platforms: MinGW, Cygwin, OpenBSD, NetBSD, xWorks RTOS, etc.
We support POSIX-compatible platforms. Do OpenBSD and NetBSD need special care?
Yes, because our support is GNU/Linux-centric. Features not supported on Linux degrade with time.
06.11.17 19:41, Victor Stinner пише:
Example of platforms: MinGW, Cygwin, OpenBSD, NetBSD, xWorks RTOS, etc.
But the way, is there an exhaustive list of platforms "officially" supported by CPython?
Several month ago there was a couple of buildbots including NetBSD and OpenBSD. What happened to them? Was the support of NetBSD and OpenBSD officially stopped as well as the support of OpenIndiana?
2017-11-07 0:41 GMT+01:00 Serhiy Storchaka
Several month ago there was a couple of buildbots including NetBSD and OpenBSD. What happened to them? Was the support of NetBSD and OpenBSD officially stopped as well as the support of OpenIndiana?
While I don't recall seeing any NetBSD buildbot, I recall that we had a OpenBSD buildbot but it was red as far as I recall. Many tests were failing on OpenBSD: test_crypt, test_socket, and many others. Since we have much more "green" buildbot workers nowadays, I would prefer to wait until almost all tests pass on a "new" platform, before plugging a new buildbot. For example, I am now subscribed to the buildbot-status@ mailing list which may be flooded by failures until a buildbot becomes stable. I'm ok with skipping enough tests for a "new" platforms to get a "green" test suite, and fix skipped tests later. In my experience, keeping tests which fail forever is annoying and hides new regressions. Even if a buildbot which always fail shouldn't send email notifications, in practice, buildbot likes to send emails for various reasons, even if the previous build was already a failure and the new build is still a failure :-) But maybe we can adjust the buildbot configuration to not send notifications on some buildbot workers. Victor
participants (5)
-
Antoine Pitrou
-
Serhiy Storchaka
-
Skip Montanaro
-
Steve Dower
-
Victor Stinner