[Distutils] (no subject)

[Zooko O'Whielacronx]

 Floris Bruynooghe <floris.bruynooghe at gmail.com> writes:
>
> In my ideal world python package developers would only ever have to
> worry about uploading an sdist.
...
> If I am correct the only reason people started to go away from this is
> because Windows users don't tend to have compilers installed.

I recently made this transition.  For Tahoe-1.2.0 I tried to make sdists
the canonical, preferred format (not, of course, omitting other formats
such as .deb's).

It turns out that using sdists as the primary, preferred format, failed
in a lot of cases.  A huge number of cases is that there is no C
compiler or no Python.h -- not only on Windows but increasingly on
Linux.  My sons's OLPCs, for example, come with Python as the standard,
integrated programming language, but not with a C compiler -- they can't
afford the storage space!

Many Python packages -- including three or four of the ones used by
Tahoe -- have optional C/C++ extension modules which are only for CPU
optimization or which are only for specific platforms or uses.  This
ticket is to change distutils so that these modules can be marked as
optional and building the package can finish even when the compilation
of the extension fails:

http://bugs.python.org/issue4706

Another large number of cases was that some Python packages require
special environmental customization, such as pyOpenSSL, which requires
that OpenSSL binaries have been installed in just the right way before
pyOpenSSL's build will work.  (You would be surprised at just how much
work it is to get OpenSSL installed in just the right way so that
pyOpenSSL can build -- the pyOpenSSL maintainers still haven't
accomplished it on Windows.)

Then there is the problem that compiling certain packages takes way too
long, such as my own pycryptopp package, which takes minutes to build
(compiling a lot of C++ files from Wei Dai's Crypto++ library).

Finally, there is the problem that a large number of users refused to
*try* the sdist install because they (more or less correctly) assumed
that it would require a lot of manual administration of build tools on
their part and would fail on their particular platform.

So, for the imminent Tahoe-1.3.0 release, I'm trying to make bdist_eggs
be the default, preferred format for any packages which include C/C++
extensions, and sdist for pure-Python packages (not, of course,
excluding other formats such as .deb's).  Here is the decentralized,
secure directory where I'm accumulating these eggs:

http://testgrid.allmydata.org:3567/uri/URI:DIR2-RO:snrfwfxatrci35zdgjnzxxx2ke:unarxv347edtku3xzmefy4mcdmfngxzeb72iyqcadbjzjpczjx5a/index.html

Now if pypi is unreachable (which seems to happen a lot recently), the
Tahoe build process will get the packages from that directory.  Also,
that URL includes the secure hash of a public key, and only authorized
uploaders have the private key, and every entry in the directory is
signed by that private key, so we can be more confident that these
packages are the ones uploaded by those authorized people.

Regards,

Zooko
---
Tahoe, the Least-Authority Filesystem -- http://allmydata.org
store your data: $10/month -- http://allmydata.com/?tracking=zsig