[Python-Dev] Update on Cygwin support (was: Clarifying Cygwin support in CPython)

[Erik Bray]

On Mon, Jul 30, 2018 at 5:26 PM Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> On 26 July 2018 at 02:13, Erik Bray <erik.m.bray at gmail.com> wrote:
> > I think a new approach that might be more practical for actually
> > getting this platform re-supported, is to go ahead and add a CI build,
> > and just skip all known failing test modules.  This is what I've done
> > in a new PR to add a Cygwin build on AppVeyor:
> >
> > https://github.com/python/cpython/pull/8463
> >
> > This is far from ideal of course, and should not mean the platform is
> > "supported".  But now I and others can go through and fix the
> > remaining test failures, re-enable those modules in the CI
> > configuration, and actually obtain some meaningful results, which will
> > hopefully encourage the core committers to accept fixes for the
> > platform.
>
> I believe the PEP 538 & 540 locale handling tests are amongst those
> that are still a bit sketchy (or outright broken?) on Cygwin, and I
> think having an advisory CI bot would definitely help with that.
> (Cygwin/MinGW are an interesting hybrid that really highlight the fact
> that neither "POSIX implies not Windows" nor "Windows implies the
> Win32 API" are entirely valid assumptions)

Yes, I believe those tests are still a little broken, though the
improvements you last made to them should be helpful in getting it
passing.  I haven't looked at it in a few months.

Indeed, it makes for some interesting broken assumptions.  Another
example I've encountered recently is because Cygwin uses the posixpath
module, all handling of Windows-style paths is broken.  This is fine,
because in general a developer should *not* be using Windows paths on
Cygwin; POSIX paths only.  However, the fact remains that Cygwin does
(mostly) transparently support Windows paths at the system level, so
some things work.  But if a user runs a script that happens to be
written in Python, but passes Windows paths to it, say, as
command-line arguments, it may or may not work.  If the path is passed
directly to open(), no problem.  But if it goes through
os.path.abspath for example things blow up.

I'm undecided as to whether this is something that developers writing
applications that support Cygwin need to handle, or if this is
something that could work better on the Python end as well.  I lean
toward the former, but I also wonder if there isn't more that could be
done in the stdlib to improve this issue as well.  In the meantime I
wrote pycygwin [1] to help with these sorts of issues in my own
software.


> So your suggested approach seems like a plausible way forward to me.
>
> The main potentially viable alternative I see would be to set up the
> *buildbot* first, and then devote the custom builder branch to the
> task of Cygwin testing for a while:
> https://devguide.python.org/buildbots/#custom-builders
>
> However, I think the overall UX of that would be worse than going down
> the advisory CI path (especially since it wouldn't really help with
> the aspect of parallel development introducing new Cygwin failures).

Exactly.  And at least for starters I might have to push to
buildbot-custom, because without a few minimal fixes in place CPython
currently does not build successfully at all on Cygwin, which makes
the buildbot a little unhelpful.

But Zach is already in touch with me about getting a buildbot worker
set up anyways.  I agree it's still good to have, and will be more and
more useful as I get the requisite fixes merged...

Thanks,
E



[1] http://pycygwin.readthedocs.io/en/latest/