[Distutils] tourist here, with a dumb RTFM question

[Jason R. Coombs]

Thanks Mike for the report. I haven't read the rest of this thread yet, and I want to respond to your OP before I do. As a result, there will likely be some duplication.

You're absolutely right to be dismayed.

I had incorrectly assessed the potential impact of this change. I had expected the impact to be minor and isolated. I underestimated the number of systems that automatically upgrade Setuptools as well as the number of packages that could be using Features. I relied too heavily on the DeprecationWarning to have flagged the breakage (to package maintainers and their users). I had expected issues like you raised in this post to have been hashed out or at least raised long before I considered the release.

I hope my decision to backout the release serves as a strong signal that this type of breakage is unacceptable. I hope the tickets filed and discussion will lead to a better mechanism for communicating these types of changes in the future. In the future, I will leverage Twitter to announce in advance any backward-incompatible releases, giving additional visibility and oversight.

I do also want to ask that deployment teams consider the implications of always upgrading to the latest release of Setuptools. I'm encouraged that they do, and I do want to keep that as a viable deployment workflow. However, at the same time, there is some inherent risk with automatically upgrading to known-backward-breaking versions. The purpose of adopting semver for versioning is to allow the project to signal directly and unambiguously when a release is backward-breaking. I haven't yet decided when or where to raise this discussion, and it certainly doesn't excuse the challenges faced Thursday.

It's clear to me from your post that there are use-cases for Features that aren't covered by simply "using extras". Namely, extras doesn't support exclusion, only inclusion. I'll make mention of that in the deprecation ticket (https://bitbucket.org/pypa/setuptools/issue/65/) and follow up after understanding the problem better.

I'm the one that needs to apologize, not you. Thanks for your patience and understanding as I and the rest of the PyPA work toward the best means of communication (that doesn't require every package maintainer to follow distutils-sig).

Best regards,
Jason

> -----Original Message-----
> From: Distutils-SIG [mailto:distutils-sig-
> bounces+jaraco=jaraco.com at python.org] On Behalf Of Michael Bayer
> Sent: Thursday, 06 March, 2014 12:56
> To: distutils-sig at python.org
> Subject: [Distutils] tourist here, with a dumb RTFM question
> 
> Hi Distutils !
> 
> I don't follow this list and haven't looked at it in a long time.   However, I'm
> learning via twitter that a brand new setuptools release that's just gone out
> has just removed the "Feature" mechanism.
> 
> Now as you're all rolling your eyes and preparing to bang out frustrated
> replies how this was well announced and deprecated and warned about and
> wow really didn't you know the term paper was due today, OK first off let me
> say I am sorry!  I am not a distutils/setuptools maintainer.    I am just
> someone that uses it, and as I include setuptools in my setup.py, I am also
> getting thousands of other people who download my product to use it as
> well.    And I don't read the setuptools changelog!  Or the setuptools blog.  Or
> this list.  I assume you guys have it under control (and you certainly do!).
> There seem to be other people like me (people who write very widely used
> software) who also seem surprised!   And that is surprising too (as I am
> usually the only person to be surprised by these things that should not be
> surprising).  So I hope you can hold back your frustration with my clueless
> RTFMness long enough to answer these questions:
> 
> 1. where was this announced?   I'm wondering why there weren't loud
> blaring blog posts and tweets all over the place stating that on March 6,
> dozens of major packages are going to all break.
> 
> 2. what is the plan for unmaintained packages and old versions of existing
> packages on pypi that "import Feature" and can no longer be installed?    Is it
> just the case that a large amount of pypi just won't install anymore?
> 
> 3. What, if any, is the recommended approach going forward to a Python
> package that wishes to specify a command-line flag during install.  Here's
> multiple choice:
> 
>     a. you can use new setuptools API <some new way to add -flags>
> 
>     b. you can roll it yourself in setup.py using <hack X>
> 
>     c. your setup.py should never accept any kind of flags as that interferes
> with <up-and-coming use case Q>
> 
>     d. other
> 
> If choice c., then here is another question.  My library includes optional C
> extensions.   On certain platforms, these C extensions don't build (like on
> Pypy, or on Windows if you don't have compilation tools installed).  In this
> regard it gracefully degrades.  But also, I want to be able to have a command
> line option to determine this as well.  Because!  Maybe I'm installing within a
> test suite where I need to test the entire library without the C extensions
> built.    Maybe some user has found a bug in the C extensions, and that user
> needs to temporarily install the tool without the extensions built.    Other
> cases for flags are, maybe your library can build with or without SSL support,
> something like that.
> 
> Keep in mind, I actually *won't* be getting bug reports about this because
> my setup.py already gracefully degrades to distutils!  But still, I'd like to have
> my -without-cextensions flag somehow.
> 
> Thanks for listening and again I apologize for not following the setuptools
> changelog on a regular basis!
> 
> 
> - mike
> 
> 
> 
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig