On Wed, Nov 8, 2017 at 3:39 PM Erik Bray <erik.m.bray@gmail.com> wrote:
Hi folks,
As some people here know I've been working off and on for a while to improve CPython's support of Cygwin. I'm motivated in part by a need to have software working on Python 3.x on Cygwin for the foreseeable future, preferably with minimal graft. (As an incidental side-effect Python's test suite--especially of system-level functionality--serves as an interesting test suite for Cygwin itself too.)
This is partly what motivated PEP 539 [1], although that PEP had the advantage of benefiting other POSIX-compatible platforms as well (and in fact was fixing an aspect of CPython that made it unfriendly to supporting other platforms).
As far as I can tell, the first commit to Python to add any kind of support for Cygwin was made by Guido (committing a contributed patch) back in 1999 [2]. Since then, bits and pieces have been added for Cygwin's benefit over time, with varying degrees of impact in terms of #ifdefs and the like (for the most part Cygwin does not require *much* in the way of special support, but it does have some differences from a "normal" POSIX-compliant platform, such as the possibility for case-insensitive filesystems and executables that end in .exe). I don't know whether it's ever been "officially supported" but someone with a longer memory of the project can comment on that. I'm not sure if it was discussed at all or not in the context of PEP 11.
I have personally put in a fair amount of effort already in either fixing issues on Cygwin (many of these issues also impact MinGW), or more often than not fixing issues in the CPython test suite on Cygwin--these are mostly tests that are broken due to invalid assumptions about the platform (for example, that there is always a "root" user with uid=0; this is not the case on Cygwin). In other cases some tests need to be skipped or worked around due to platform-specific bugs, and Cygwin is hardly the only case of this in the test suite.
I also have an experimental AppVeyor configuration for running the tests on Cygwin [3], as well as an experimental buildbot (not available on the internet, but working). These currently rely on a custom branch that includes fixes needed for the test suite to run to completion without crashing or hanging (e.g. https://bugs.python.org/issue31885). It would be nice to add this as an official buildbot, but I'm not sure if it makes sense to do that until it's "green", or at least not crashing. I have several other patches to the tests toward this goal, and am currently down to ~22 tests failing.
Before I do any more work on this, however, it would be best to once and for all clarify the support for Cygwin in CPython, as it has never been "officially supported" nor unsupported--this way we can avoid having this discussion every time a patch related to Cygwin comes up. I could provide some arguments for why I believe Cygwin should supported, but before this gets too long I'd just like to float the idea of having the discussion in the first place. It's also not exactly clear to me how to meet the standards in PEP 11 for supporting a platform--in particular it's not clear when a buildbot is considered "stable", or how to achieve that without getting necessary fixes merged into the main branch in the first place.
Thanks, Erik
[1] https://www.python.org/dev/peps/pep-0539/ [2] https://github.com/python/cpython/commit/717d1fdf2acbef5e6b47d9b4dcf48ef1829... [3] https://ci.appveyor.com/project/embray/cpython
Apologies for responding to a months old post, but rather than repeat myself verbatim I'll just mention that all of the above is still true and relevant, and I am still interested in getting Python somewhere closer to "stable" on Cygwin. Part of the problem with my previous approach is that I was trying to fix every last test failure before asking to add Cygwin to CPython's CI fleet. While I believe all failures *should* be fixed (or skipped as appropriate) this is not practical to do in a short amount of time, and not having CI implemented for a platform means new bugs are added faster than we can fix the existing bugs. For example, between 3.6 and 3.7 two new bugs have caused Python to be unbuildable on Cygwin: https://bugs.python.org/issue34211 https://bugs.python.org/issue34212 This is in addition to an older issue that I was hoping to have fixed in Python 3.7, as the PR was "green" for some time well before its release: https://github.com/python/cpython/pull/4348 However, I fear part of why it was never merged is that lack of CI for Cygwin, which is in turn because it was impossible to get a "stable" test pass on Cygwin in the first place--something of a catch-22. I think a new approach that might be more practical for actually getting this platform re-supported, is to go ahead and add a CI build, and just skip all known failing test modules. This is what I've done in a new PR to add a Cygwin build on AppVeyor: https://github.com/python/cpython/pull/8463 This is far from ideal of course, and should not mean the platform is "supported". But now I and others can go through and fix the remaining test failures, re-enable those modules in the CI configuration, and actually obtain some meaningful results, which will hopefully encourage the core committers to accept fixes for the platform. Once a few more major fixes are accepted (there are many for which I have fixes, but have not yet made PRs) I can also add a buildbot for Cygwin. I already had a working experimental buildbot last year, so now it's just a matter of getting a few critical issues (such as the above-mentioned build issues) fixed, plus more of the major test modules. Thanks for your consideration, E