[Python-Dev] Packaging and binary distributions for Python 3.3

[Paul Moore]

I see that the Packaging documentation is now more complete (at least
at docs.python.org) - I don't know if it's deemed fully complete yet,
but I scanned the documentation and "Installing Python Projects" looks
pretty much converted (and very good!!), but "Distributing Python
Projects" still has quite a lot of distutils-related text in, and I
need to read more deeply to understand if that's because it remains
unchanged, or if it is still to be updated.

But one thing struck me - the "Installing Python Projects" document
talks about source distributions, but not much about binary
distributions.

On Windows, binary distributions are significantly more important than
on Unix, because not all users have easy access to a compiler, and
more importantly, C library dependencies can be difficult to build,
hard to set up, and generally a pain to deal with. The traditional
solution was always bdist_wininst installers, and with the advent of
setuptools binary eggs started to become common. I've noticed that
since pip became more widely used, with its focus on source builds,
binary eggs seemed to fade away somewhat. I don't know what format
packaging favours.

The problem when Python 3.3 comes out is that bdist_wininst/bdist_msi
installers do not interact well with pysetup. And if native virtual
environment support becomes part of Python 3.3, they won't work well
there either (they don't deal well with today's virtualenv, for that
matter). So there will be a need for a pysetup-friendly binary format.
I assume that the egg format will fill this role - or is that not the
case? What is the current thinking on binary distribution formats for
Python 3.3?

The main reason I am asking is that I would like to write an article
(or maybe a series of articles) for Python Insider, introducing the
new packaging facilities from the point of view of an end user with
straightforward needs (whether a package user just looking to manage a
set of installed packages, or a module author who just wants to
publish his code in a form that satisfies as many people as possible).
What I'd hope to do is, as well as showing people all the nice things
they can expect to see in Python 3.3, to also start package authors
thinking about what they need to do to support their users under the
new system. If we get the message out early, and make people aware of
the benefits of the new end user tools, then I'm hoping more authors
will see the advantage of switching to the new format rather than just
sticking with bdist_xxx because "it's always worked".

I suspect I should (re-)join the distutils SIG and take this
discussion there. But honestly, I'm not sure I have the time - the
traffic was always fairly high, and the number of relevant posts for a
casual observer was quite low. So even if that's the right place to
go, some pointers to some "high spots" to get me up to speed on the
current state of affairs would help.

Thanks,
Paul.