[Distutils] Multi-version import support for wheel files

[PJ Eby]

On Mon, Aug 26, 2013 at 11:15 AM, Donald Stufft <donald at stufft.io> wrote:
> There is always a cost. In this case mostly in complexity and start up time.
>
> As you mentioned originally the cost to multi version support was the need
> to use a require() function and when people complained about that you
> added the .pth files which imposed another sort of cost to people using
> multi versioned installs.

See, this is exactly what I'm talking about: you've got this 100% backwards:

.pth files are for people who *aren't* using multi-version imports.
They're for *default* versions, not alternate versions!

And they're utterly unnecessary for Nick's proposal.


> You claim it is part of core Python but it's really not, if it was it wouldn't require
> importing pkg_resources of the .pth files to make it work.

As I pointed out in the email you apparently didn't read, along with
multiple emails from Jim: pkg_resources isn't necessary for
alternate-version support.  All that's required for alternate versions
is to add them to sys.path, which buildout does just fine *without
pkg_resources*.


> I find it ridiculous that you'd call this thread 90% FUD when the vast bulk of the
> thread has been trying to determine if there were any reasonable concerns
> with the approach and upon examination determined that the biggest problem
> with it was attaching it to Wheel and not the multi version support at all

What I'm referring to as the FUD is that people have been confusing
what Nick proposed with what setuptools does, and getting *both* of
them wrong in the details.

Nick's proposal was not to mimic setuptools' multi-version support,
but rather to provide something else: let's call it "alternate version
support", to separate it from what setuptools does.

In Nick's AVS proposal, there is *no* overhead for anything that
doesn't need a non-default version, and it's 100% opt-in, used only
for things that need *non-default* versions.

Note, by the way, that since these *non-default* packages aren't on
sys.path by default, *there is no overhead and no .pth files are
involved*.  They are effectively invisible and irrelevant for anything
that doesn't use them.

The only place where there's overhead is in the script that needs the
alternative version(s), and its sys.path is lengthened only by those
items that it can't obtain from the default sys.path.  And if you use
buildout's approach of simply adding:

     sys.path[0:0] = [path1,...]

to the head of a script, then *pkg_resources isn't involved either*.

This is bog-standard stock Python.

So the FUD part I was referring to is all the "oh no, setuptools is
complicated" in response to Nick's perfectly reasonable idea *which
doesn't involve any of setuptools' complexity*, because it's doing
something completely different.


> I realize
> setuptools and easy_install are your baby but the persecution complex doesn't
> help to win people over to your side of things.

I think you're confused here.  I don't think setuptools is being
persecuted, I think *Nick's idea* is being misunderstood, and being
construed as almost the exact *opposite* of what it is.

All the stuff people bitch about that relates to multi-versions in
setuptools are actually issues with setuptools' implementation of
*default* versions, not *alternative* versions.  So to look at Nick's
proposal and think it's going to have the same problems is completely
ludicrous - it's 180 degrees opposite of what setuptools does, because
for setuptools, *default versions* are the special case -- they're
what cause 90% of the complexity in pkg_resources' manipulation of
sys.path, and they're the main reason .pth files are ever used.

So it's crazy-making to see people thinking Nick's proposal is going
to bring all that crap along, when that's the exact *opposite* of the
situation.


> In my experience setuptools has a lot of good ideas but they are wrapped in bad
> ideas or implementations that obscure the fact that there *are* good ideas there.
> I do not believe it to be unreasonable for people to want to make sure that we're
> standardizing around one of the *good* ideas instead of one of the bad ideas.

It would help if people understood the actual facts, then.  AFAICT,
Nick's proposal doesn't do any of the things that people are worried
about, or at the very least does not *require* them.  As Jim and I
have pointed out more than once, pkg_resources is not a runtime
requirement to allow alternative versions to be importable by code
that wants them.

It would really be a shame to shoot down Nick's idea based on a vague
misunderstanding of it.  It's a good proposal, and has far less to do
with setuptools than most people in the thread seem to think.