[Distutils] Splitting large packages into multiple eggs
Phillip J. Eby
pje at telecommunity.com
Wed Dec 14 23:16:50 CET 2005
At 07:02 PM 12/11/2005 -0800, Bob Ippolito wrote:
>It looks like easy_install can't do this yet.
And it's not going to be able to. The closest thing I can envision is that
if you have a master project with a setup.py that runs each child project's
setup.py (or just a setup() call), such that each gets passed a "bdist_egg"
command by easy_install. EasyInstall already detects when multiple eggs
are built by a single setup.py, and processes all of them.
That way, you could have one PyObjC project that contains all the others,
and builds multiple eggs, including one that just specifies dependencies on
The only possible issue that might arise is if there are inter-egg
dependencies and the eggs are built out-of-order. In that case,
easy_install might incorrectly conclude that it needs one of the built
eggs, before it has processed it. I could probably add some code to make
this more robust.
Anyway, the only limitation of this approach is that you won't be able to
build any individual packages from source, only the overall package as a
whole. However, it will be possible to build and even upload all the eggs
using "setup.py bdist_egg upload", so easy_install will be able to find the
binary packages. Or you can just have your PyPI download URL point to a
directory where you dump the latest eggs; if you're running your bdist
commands on the web server (or rsync it), you can use the 'rotate' command
to delete outdated snapshots.
But anybody who wants to build from source will have to use the master
package, rather than any individual subpackages.
>Also, I'm going to want a way to have "setup.py develop" ensure that
>all of the subprojects are up to date.
This should work normally if your setup.py just calls setup() for each
subproject and the master; again this should be in dependencies-first order.
More information about the Distutils-SIG