[Distutils] Oddities in setuptools/distribute

[P.J. Eby]

At 04:31 PM 10/5/2010 -0400, Barry Warsaw wrote:
>I'm working on some patches that allow for multiple builds of Python with
>different options to coexist.  This is an extension to PEP 3149 and has been
>discussed recently in python-dev:
>
>http://mail.python.org/pipermail/python-dev/2010-October/104382.html
>
>This has led me into the twisty maze of setuptools and distribute:
>
>http://mail.python.org/pipermail/python-dev/2010-October/104430.html
>
>I believe I've figured out a patch that *should* make this work, but doesn't,
>and I have a few questions about some things I've noticed:
>
>1) Why does setuptools write a stub loader for the .so in the first place?
>    Why not just let the import of the shared library happen 
> directly using the
>    normal Python import rules?

It does.  The normal import rules load .so files first, so the stub 
loaders only have an effect if the module is imported from a 
*zipfile* (where .so files won't normally load at all).

When unzipped, the .py file is ignored, and so has no effect.  But 
when zipped, it forces extraction of the .so to a cache directory, 
where it can then be imported.


>2) Why does build_ext.py and bdist_egg.py have similar but slightly different
>    stub writers?
>
>    I'm sure there's an important reason for this, but at least in the case of
>    shared library loading, it seems like the two stubs ought to be more
>    similar than they are.

Dynamic shared library support in setuptools is still officially an 
experimental feature, so there's some possibility that this is a 
bug.  That is, bdist_egg's stubs may be broken for extensions using 
dynamic library loader stubs.


>3) Why does site-packages/<package>.egg/ get deleted when the package is
>    re-installed?

Because if you're reinstalling it, it's presumably because the 
original is fux0red in some manner.  Also, depending on your 
command-line options, you might be installing an .egg file where a 
directory existed before, or vice versa.


>   Can this be prevented?
>
>    This is actually the show-stopper I'm stuck on because if I install my
>    extension with build-A of Python, then try to install it with build-B of
>    Python, the build-A version gets wiped.  Because the shared libraries now
>    have build-flag discriminators in the file name, the two installs *should*
>    be able to coexist,

On a version of Python that does this, it would probably be best if 
the platform string included the build flags -- then you would have 
two separate .eggs, each of which would only be loaded by a compatible runtime.


>Thanks for any feedback you can provide.

Thanks for asking!