[Distutils] Parallel installation of incompatible versions

PJ Eby pje at telecommunity.com
Tue Mar 19 19:30:36 CET 2013

On Tue, Mar 19, 2013 at 1:02 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> The "but what
> about long sys.paths?" problem can be dealt with as a performance
> issue for the import system.

And already has been, actually.  ;-)

In addition to the changes made in the import system for 3.3, there's
another improvement possible, relative to today's easy_install-based
path lengths.

Currently, easy_install dumps everything into subdirectories, so
sys.path is *always* long for the default case, and *just* as long for
non-default apps.

However, if instead of just listing default versions in
easy-install.pth, those versions were actually installed PEP
376-style, and only the non-default versions had to be added to
sys.path for the apps that need them, then you'd see a dramatic
shortening of sys.path for *all* apps and scenarios.

Today, if you have N default libraries and an average of M non-default
libraries per app, plus C as  the constant minimum sys.path length,
then sys.path is N+C by default, and N+C+M for each app that uses
non-default libraries.

But, under a hybrid scheme of PEP 376 for defaults plus an extension
for non-defaults, the default sys.path length is C, and C+M for the
apps needing non-default versions.

In other words, N disappears -- and "N" is usually a lot bigger than C or M.

TBH, if I had access to the time machine right now, easy_install would
have worked this way from the start, instead of using easy-install.pth
as a version-switching mechanism.  (The main reason it didn't work out
that way to begin with, is because the .egg-info concept wasn't
invented until much later in easy_install's development.)

More information about the Distutils-SIG mailing list