[Python-Dev] Raising objections

[Phillip J. Eby]

At 04:15 PM 4/19/2006 -0400, A.M. Kuchling wrote:
>At least some of these changes to Distutils seem unobjectionable for
>inclusion.
>
>For example, the changes to Command just allow keyword arguments on
>two methods and adds a class attribute; they seem unlikely to break
>any existing users of the class.
>
>The Extension change replaces .pyx source files with .c; I'm not sure
>what the purpose of this change is, but it clearly might cause
>problems for distributions with Pyrex source files.

The purpose of the extension change is that it first checks whether Pyrex's 
distutils extension is available.  If it is not available, it changes the 
source file names to .c, so that any pre-processed Pyrex files will be 
processed.  This allows Pyrex-using packages to include pre-translations 
from Pyrex to C, and non-Pyrex users can still compile the package.

For example, PyProtocols ships with a _speedups.pyx file, and a _speedups.c 
file, produced locally with Pyrex.  Let's say a user is building 
PyProtocols from source.  If they have Pyrex installed, setuptools uses 
Pyrex to rebuild the .c from the .pyx.  If they do *not* have Pyrex 
installed, setuptools tells distutils to just build directly from the .c 
file, so that the distutils isn't confused by all this '.pyx' business.


>The Distribution changes add some more optional kw arguments -- no
>problem there -- and a bunch of egg-specific methods.  This set would
>need some further study, and also bakes in .egg support; we'd have to
>be very sure that .eggs are the way to go, so this might need a PEP
>and/or BDFL pronouncement.

There's a slightly deeper issue there than just egg support itself.  It's 
*plugin* support.  Setuptools allows eggs to register plugins to provide 
new commands or keyword options that then apply either globally or only to 
specific setup scripts.  Setuptools then uses this mechanism to register 
its own commands as distutils replacements.

What this means is that if setuptools is installed, and you use setuptools' 
Distribution class, then setuptools' command plugins will apply -- which 
means that you now have setuptools behavior for those commands, instead of 
distutils default behavior.  And this would be the case even if you never 
imported setuptools, if you just ported the Distribution class over.

So, you can't actually activate the plugin support in setuptools' 
Distribution for distutils, or you'll be removing backward 
compatibility.  There would have to be some way to specify whether 
setuptools or distutils should take precedence in the event that both 
define a given command.  For example, a 'use_setuptools' argument that 
defaults to false, unless you first imported setuptools (for back/side 
compatibility w/existing setuptools scripts).


>Obviously, applying changes to Distutils makes Phillip's maintenance
>of setuptools more difficult -- now he has to worry about
>monkeypatching or not depending on the Python version.

Yeah, there's already some of this to deal with for package 
data.  Setuptools was added to the sandbox in 2004, and shortly afterwards, 
Fred Drake took on the task of adding some of its features to distutils for 
Python 2.4.  Which was great, except that then setuptools had to start 
worrying about whether or not the stuff it was subclassing was in fact 
already doing what it was trying to do.  So, there's a bunch of fragile 
conditional code in there, and I'm not thrilled at the idea of adding more, 
because it significantly increases the number of testing combinations 
required to ensure that things work when changes are made, as well as 
making it harder to tell what's going on in the code.


>   But at least
>some of the monkeypatching can be removed -- maybe all of it if the
>Distribution class proves amenable.

The Distribution class also adds the ability for individual commands to 
have positional argument parsing.