As some people here know I've been working off and on for a while to
improve CPython's support of Cygwin. I'm motivated in part by a need
to have software working on Python 3.x on Cygwin for the foreseeable
future, preferably with minimal graft. (As an incidental side-effect
Python's test suite--especially of system-level functionality--serves
as an interesting test suite for Cygwin itself too.)

This is partly what motivated PEP 539 [1], although that PEP had the
advantage of benefiting other POSIX-compatible platforms as well (and
in fact was fixing an aspect of CPython that made it unfriendly to
supporting other platforms).

As far as I can tell, the first commit to Python to add any kind of
support for Cygwin was made by Guido (committing a contributed patch)
back in 1999 [2]. Since then, bits and pieces have been added for
Cygwin's benefit over time, with varying degrees of impact in terms of

# ifdefs and the like (for the most part Cygwin does not require much

in the way of special support, but it does have some differences from
a "normal" POSIX-compliant platform, such as the possibility for
case-insensitive filesystems and executables that end in .exe). I
don't know whether it's ever been "officially supported" but someone
with a longer memory of the project can comment on that. I'm not sure
if it was discussed at all or not in the context of PEP 11.

I have personally put in a fair amount of effort already in either
fixing issues on Cygwin (many of these issues also impact MinGW), or
more often than not fixing issues in the CPython test suite on
Cygwin--these are mostly tests that are broken due to invalid
assumptions about the platform (for example, that there is always a
"root" user with uid=0; this is not the case on Cygwin). In other
cases some tests need to be skipped or worked around due to
platform-specific bugs, and Cygwin is hardly the only case of this in
the test suite.

I also have an experimental AppVeyor configuration for running the
tests on Cygwin [3], as well as an experimental buildbot (not
available on the internet, but working). These currently rely on a
custom branch that includes fixes needed for the test suite to run to
completion without crashing or hanging (e.g.
https://bugs.python.org/issue31885). It
would be nice to add this as
an official buildbot, but I'm not sure if it makes sense to do that
until it's "green", or at least not crashing. I have several other
patches to the tests toward this goal, and am currently down to ~22
tests failing.

Before I do any more work on this, however, it would be best to once
and for all clarify the support for Cygwin in CPython, as it has never
been "officially supported" nor unsupported--this way we can avoid
having this discussion every time a patch related to Cygwin comes up.
I could provide some arguments for why I believe Cygwin should
supported, but before this gets too long I'd just like to float the
idea of having the discussion in the first place. It's also not
exactly clear to me how to meet the standards in PEP 11 for supporting
a platform--in particular it's not clear when a buildbot is considered
"stable", or how to achieve that without getting necessary fixes
merged into the main branch in the first place.

a platform--in particular it's not clear when a
buildbot is considered
"stable", or how to achieve that without getting necessary fixes
merged into the main branch in the first place.

I think in this context, "stable" just means "keeps a connection to
the buildbot master and doesn't blow up when told to build" :). As
such, I'm ready to get you added to the fleet whenever you are.

Yep, and from a PEP 11 perspective, it's fine to have the order be:

add the new buildbot worker while it's still red

make the changes needed to turn it green in 3.X

keep it green for future releases of 3.X+

There just needs to be enough lead time between step 1 and the release
of 3.Xb1 to land the compatibility fixes without causing problems for
the release manager (and you're currently fine on that front with
respect to 3.7).

a platform--in particular it's not clear when a
buildbot is considered
"stable", or how to achieve that without getting necessary fixes
merged into the main branch in the first place.

I think in this context, "stable" just means "keeps a connection to
the buildbot master and doesn't blow up when told to build" :). As
such, I'm ready to get you added to the fleet whenever you are.

"Doesn't blow up when told to build" is the tricky part, because there
are a few tests that are known to cause the test suite process to hang
until killed. It's not clear to me whether, even with the --timeout
option, that the test runner will kill hanging processes (I haven't
actually tried this though so I'll double-check, but I'm pretty sure
it does not).

So until at least those issues are resolved I'd be hesitate to call it "stable".

As some people here know I've been working off and on for a while to
improve CPython's support of Cygwin. I'm motivated in part by a need
to have software working on Python 3.x on Cygwin for the foreseeable
future, preferably with minimal graft. (As an incidental side-effect
Python's test suite--especially of system-level functionality--serves
as an interesting test suite for Cygwin itself too.)

This is partly what motivated PEP 539 [1], although that PEP had the
advantage of benefiting other POSIX-compatible platforms as well (and
in fact was fixing an aspect of CPython that made it unfriendly to
supporting other platforms).

As far as I can tell, the first commit to Python to add any kind of
support for Cygwin was made by Guido (committing a contributed patch)
back in 1999 [2]. Since then, bits and pieces have been added for
Cygwin's benefit over time, with varying degrees of impact in terms of

# ifdefs and the like (for the most part Cygwin does not require much

in the way of special support, but it does have some differences from
a "normal" POSIX-compliant platform, such as the possibility for
case-insensitive filesystems and executables that end in .exe). I
don't know whether it's ever been "officially supported" but someone
with a longer memory of the project can comment on that. I'm not sure
if it was discussed at all or not in the context of PEP 11.

I have personally put in a fair amount of effort already in either
fixing issues on Cygwin (many of these issues also impact MinGW), or
more often than not fixing issues in the CPython test suite on
Cygwin--these are mostly tests that are broken due to invalid
assumptions about the platform (for example, that there is always a
"root" user with uid=0; this is not the case on Cygwin). In other
cases some tests need to be skipped or worked around due to
platform-specific bugs, and Cygwin is hardly the only case of this in
the test suite.

I also have an experimental AppVeyor configuration for running the
tests on Cygwin [3], as well as an experimental buildbot (not
available on the internet, but working). These currently rely on a
custom branch that includes fixes needed for the test suite to run to
completion without crashing or hanging (e.g.
https://bugs.python.org/issue31885). It
would be nice to add this as
an official buildbot, but I'm not sure if it makes sense to do that
until it's "green", or at least not crashing. I have several other
patches to the tests toward this goal, and am currently down to ~22
tests failing.

Before I do any more work on this, however, it would be best to once
and for all clarify the support for Cygwin in CPython, as it has never
been "officially supported" nor unsupported--this way we can avoid
having this discussion every time a patch related to Cygwin comes up.
I could provide some arguments for why I believe Cygwin should
supported, but before this gets too long I'd just like to float the
idea of having the discussion in the first place. It's also not
exactly clear to me how to meet the standards in PEP 11 for supporting
a platform--in particular it's not clear when a buildbot is considered
"stable", or how to achieve that without getting necessary fixes
merged into the main branch in the first place.

Apologies for responding to a months old post, but rather than repeat
myself verbatim I'll just mention that all of the above is still true
and relevant, and I am still interested in getting Python somewhere
closer to "stable" on Cygwin.

Part of the problem with my previous approach is that I was trying to
fix every last test failure before asking to add Cygwin to CPython's
CI fleet. While I believe all failures should be fixed (or skipped
as appropriate) this is not practical to do in a short amount of time,
and not having CI implemented for a platform means new bugs are added
faster than we can fix the existing bugs. For example, between 3.6
and 3.7 two new bugs have caused Python to be unbuildable on Cygwin:

However, I fear part of why it was never merged is that lack of CI for
Cygwin, which is in turn because it was impossible to get a "stable"
test pass on Cygwin in the first place--something of a catch-22.

I think a new approach that might be more practical for actually
getting this platform re-supported, is to go ahead and add a CI build,
and just skip all known failing test modules. This is what I've done
in a new PR to add a Cygwin build on AppVeyor:

This is far from ideal of course, and should not mean the platform is
"supported". But now I and others can go through and fix the
remaining test failures, re-enable those modules in the CI
configuration, and actually obtain some meaningful results, which will
hopefully encourage the core committers to accept fixes for the
platform.

Once a few more major fixes are accepted (there are many for which I
have fixes, but have not yet made PRs) I can also add a buildbot for
Cygwin. I already had a working experimental buildbot last year, so
now it's just a matter of getting a few critical issues (such as the
above-mentioned build issues) fixed, plus more of the major test
modules.

I think a new approach that might be more practical
for actually
getting this platform re-supported, is to go ahead and add a CI build,
and just skip all known failing test modules. This is what I've done
in a new PR to add a Cygwin build on AppVeyor:

This is far from ideal of course, and should not mean the platform is
"supported". But now I and others can go through and fix the
remaining test failures, re-enable those modules in the CI
configuration, and actually obtain some meaningful results, which will
hopefully encourage the core committers to accept fixes for the
platform.

I believe the PEP 538 & 540 locale handling tests are amongst those
that are still a bit sketchy (or outright broken?) on Cygwin, and I
think having an advisory CI bot would definitely help with that.
(Cygwin/MinGW are an interesting hybrid that really highlight the fact
that neither "POSIX implies not Windows" nor "Windows implies the
Win32 API" are entirely valid assumptions)

However, I think the overall UX of that would be worse than going down
the advisory CI path (especially since it wouldn't really help with
the aspect of parallel development introducing new Cygwin failures).

I think a new approach that might be more
practical for actually
getting this platform re-supported, is to go ahead and add a CI build,
and just skip all known failing test modules. This is what I've done
in a new PR to add a Cygwin build on AppVeyor:

This is far from ideal of course, and should not mean the platform is
"supported". But now I and others can go through and fix the
remaining test failures, re-enable those modules in the CI
configuration, and actually obtain some meaningful results, which will
hopefully encourage the core committers to accept fixes for the
platform.

I believe the PEP 538 & 540 locale handling tests are amongst those
that are still a bit sketchy (or outright broken?) on Cygwin, and I
think having an advisory CI bot would definitely help with that.
(Cygwin/MinGW are an interesting hybrid that really highlight the fact
that neither "POSIX implies not Windows" nor "Windows implies the
Win32 API" are entirely valid assumptions)

Yes, I believe those tests are still a little broken, though the
improvements you last made to them should be helpful in getting it
passing. I haven't looked at it in a few months.

Indeed, it makes for some interesting broken assumptions. Another
example I've encountered recently is because Cygwin uses the posixpath
module, all handling of Windows-style paths is broken. This is fine,
because in general a developer should not be using Windows paths on
Cygwin; POSIX paths only. However, the fact remains that Cygwin does
(mostly) transparently support Windows paths at the system level, so
some things work. But if a user runs a script that happens to be
written in Python, but passes Windows paths to it, say, as
command-line arguments, it may or may not work. If the path is passed
directly to open(), no problem. But if it goes through
os.path.abspath for example things blow up.

I'm undecided as to whether this is something that developers writing
applications that support Cygwin need to handle, or if this is
something that could work better on the Python end as well. I lean
toward the former, but I also wonder if there isn't more that could be
done in the stdlib to improve this issue as well. In the meantime I
wrote pycygwin [1] to help with these sorts of issues in my own
software.

However, I think the overall UX of that would be worse than going down
the advisory CI path (especially since it wouldn't really help with
the aspect of parallel development introducing new Cygwin failures).

Exactly. And at least for starters I might have to push to
buildbot-custom, because without a few minimal fixes in place CPython
currently does not build successfully at all on Cygwin, which makes
the buildbot a little unhelpful.

But Zach is already in touch with me about getting a buildbot worker
set up anyways. I agree it's still good to have, and will be more and
more useful as I get the requisite fixes merged...