[Distutils] RFC: Binary Distribution Format for distutils2/packaging

Paul Moore p.f.moore at gmail.com
Thu Mar 15 00:29:12 CET 2012

On 14 March 2012 19:04, Tarek Ziadé <tarek at ziade.org> wrote:
>> Please can we have a new format that only has a Python version in the
>> filename if it matters?
> isn't that supposed to be the source release ?

Yes, basically - at least as far as I understand.

> Why would someone create a binary release when
> it's pure Python ?

I wish I knew. But people do - mostly egg format files. But I think
this is partly because of the confusion between
egg-as-distribution-format vs egg-as-directly-usable-object that PJE
alludes to in his emails.

If there's a binary egg available, how do I know whether it's needed
without trying a source install and seeing if it works? And what about
where it does, but doesn't install C accelerators because I have no
compiler? The source works, but the binary is better. How would an
automated installation process know? Binary installers for pure-Python
projects exist, and cause confusion. (Example - IPython, see

> or are you thinking about the auto-installable feature
> for windows here ?

No. I used to care about this, but no longer. I'm happy now to install
anything that *can* be installed from source directly, using pip (or
pysetup, once it works).

> I think there are two different uses cases:
> - install a program or a lib manually -- where an auto-installer makes
> sense.
> - have a library installed by a tool, where an auto-installer is not really
> needed.

I'm not sure I'd characterise the cases like that, I tend to think of
them as "wants to integrate with system standard policies" vs "wants
to use Python specific approaches". But it's the same 2 cases. I used
to be in the first camp, but I've switched and these days I'm firmly
in the second.

> I was thinking bdist_wininst and bdist_msi would stay around for people that
> want to create self-installable archives,
> and we would discuss a new format that is not self-installable.

The key thing for me, as a user, is having packages that I cannot
build myself available to me in a format that I can use. And
secondarily to have an easy user experience. The following are
possible installation scenarios:

1. pip/pysetup install PKG - transparent and simple, for source
distributions that are pure-Python or I can build (no external
dependencies and I have C installed)
2. pip/pysetup install PKG - new-format supplied by author (I assume
the install command will auto-download the new format)
3. download egg/wininst file
   run converter to create new-format
   pip/pysetup install new-format-file
   tidy up the temporary files
   - egg or wininst supplied by author, assuming converters exist and
are supplied with Python (add a "find the converter" step if they are
3rd party)
4. Install as system installer - MSI supplied by author, and I'm
installing in system Python.
5. Cry - MSI supplied by author and I want to install in a virtualenv
or similar. Basically, it's impossible (or at least, not worth the

To give me the best user experience, authors have to start supplying
the new format. The hard part is getting authors to take up a new
format (after all, they probably want to keep the old format as well,
for users who haven't switched to pysetup, so you've just doubled
their work). Alternatively, integrate the converters directly into
pip/pysetup, so that eggs and wininsts are *transparently* converted
on download, making case 3 look just like case 2. But at that point,
what's the argument for authors distributing the new format, when the
old ones work just as well for new-style users?

The technical issues aren't hard, it's the social issues that need
careful consideration.


More information about the Distutils-SIG mailing list