[Distutils] Multi-version import support for wheel files

Mon Aug 26 19:48:57 CEST 2013

Just to add a bit more "FUD" ;-)

I do a lot of packaging things up with py2app, py2exe, etc. -- I find
I often want to be able to give folks "one thing" that they can
install and run, and I'd rather they don't even need to know it's
built with python.

A while back, when I was doing this with a web app (actually a
"Browser Interface, Local Server (BILS) app --i.e, a web server and
html  interface, but the browser embedded in a wx app so it acted like
a desktop app to the user...) setuptools, and pkg_resources really
made it hard.

I don't remember the details at this point, but it was built on
Pylons, which made heavy use of setuptools at the time. And
easy_install pylons worked great. But bundling it up was a pain -- we
had to put an enormous amount of crap in there to satisfy
pkg_resources, and who knows what -- crap that was not used, and
should not have been needed at run time. In fact, we needed to put in
at least the skeleton of stuff that was a different version than the
one used.

Note also that the only reliable way to get an egg-installed package
to work with py2app was to include the whole egg.

My conclusion at the time was that there was a real confusion between
what should happen at install time, and what should happen at run
time. For instance, making sure that a dependency is there and the
right version seems like an install-time problem to me -- and yet,
there is was, getting checked every time you started up the app. I'm
not clear at this point how much of that was inherent to how
setuptools works, and how much was how Pylons chose to use it, but I
hope we'll all keep this use-case in mind at this point.

i.e. if a user has only needs one version of a given package
installed, lets not have much overhead there to support that, and
let's not require much run-time support at all.

I note that I've been a user of wx.version for wxPython, and it was
always easy and painless, including py2app, py2exe use. So it can be
done.

-Chris

On Mon, Aug 26, 2013 at 8:15 AM, Donald Stufft <donald at stufft.io> wrote:
>
> On Aug 26, 2013, at 10:33 AM, PJ Eby <pje at telecommunity.com> wrote:
>
>> On Mon, Aug 26, 2013 at 5:20 AM, Paul Moore <p.f.moore at gmail.com> wrote:
>>> On 25 August 2013 23:14, PJ Eby <pje at telecommunity.com> wrote:
>>>> Thus, you don't have to know you have multiple versions installed; it
>>>> can trivially happen by way of dependencies you aren't paying
>>>> attention to.  The more things you install, the more likely it is you
>>>> have two versions hanging around.
>>>
>>>
>>> OK, I see. But I'm not sure if we're agreeing or disagreeing over the
>>> result. To me, this is a bad thing on the principle that there is a cost to
>>> multiversion support (it's not part of core Python, so you have to do
>>> *something* to make it work)
>>
>> Seriously?  The basic functionality of using sys.path to have multiple
>> versions *is* part of core Python, and has been since 1.5.2 (16 years
>> ago), and probably longer than that.
>>
>> In the days before easy_install and virtualenv, if you needed
>> different versions of things, you used "setup.py install" to different
>> directories (assuming distutils was involved, otherwise you just
>> copied files) and either put your scripts in the same directories, or
>> used PYTHONPATH or explicit sys.path manipulation.
>>
>> That is all easy_install does: add a naming convention for the
>> directories, and automate the sys.path manipulation.
>>
>> Buildout does the same thing, it just writes the sys.path manipulation
>> into the scripts statically, instead of using pkg_resources at
>> runtime.
>>
>> So the notion of "cost" doesn't make any sense.  Tools like
>> easy_install and buildout *reduce* the management cost, they don't add
>> anything to core Python.
>>
>> (Now, if you're talking about the .pth files from easy_install, those
>> are something that I added because people complained about having to
>> use require(), and wanted to have a default version available in the
>> interpreter.)
>>
>>
>>> and so having people inadvertently pay that
>>> cost to use a feature that they don't actually *need* is wrong.
>>
>> What cost are you talking about here?  Given that most people don't
>> even know they *have* multiple versions installed or care, how is a
>> cost being imposed upon them?  Are you talking about disk storage?
>>
>>
>>> One other point, just as a matter of curiosity (because it's not relevant to
>>> the current discussion): in your explanation above, there doesn't seem to be
>>> any step that says the user normally uses CherryPy 3 (so that would be the
>>> one they would get automatically at the interactive interpreter).
>>
>> If they easy_install that version, sure, that's what they'll get as a
>> default version.
>>
>>
>>> For me,
>>> that's really the only use case I'd have for multi-versioning - 99% of the
>>> time I use a particular version of a project, but I have one particular
>>> application that can't work with the version I prefer.
>>
>> Yes, and that's the sort of scenario Nick was proposing pip support,
>> that you have an explicit "install me a different version for my other
>> app" capability -- such that that app's script wrapper adds its
>> alternate version to sys.path ahead of the default one.  So it would
>> have been opt-in and impose the "cost" of a slightly longer sys.path
>> and increased disk space usage only on those who ask for it.
>>
>> (Honestly, 90% of this entire thread has sounded like complete FUD to
>> me, i.e. fear based on a lack of understanding that there actually
>> isn't anything magical about multi-version support.  As Jim has
>> pointed out, buildout does multi-version support without even using
>> pkg_resources.  And before all these tools existed, people just
>> installed things in different directories and used either adjacent
>> scripts, PYTHONPATH, or explicit sys.path manipulation.  There is
>> nothing magical whatsoever about having multiple versions of a thing
>> installed on your system; all the tools do is add naming conventions
>> for where stuff is installed...  and having such naming conventions is
>> a *good* thing, compared to the old days.)
>
> There is always a cost. In this case mostly in complexity and start up time.
>
> As you mentioned originally the cost to multi version support was the need
> to use a require() function and when people complained about that you
> added the .pth files which imposed another sort of cost to people using
> multi versioned installs.
>
> You claim it is part of core Python but it's really not, if it was it wouldn't require
> importing pkg_resources of the .pth files to make it work.
>
> I find it ridiculous that you'd call this thread 90% FUD when the vast bulk of the
> thread has been trying to determine if there were any reasonable concerns
> with the approach and upon examination determined that the biggest problem
> with it was attaching it to Wheel and not the multi version support at all. I realize
> setuptools and easy_install are your baby but the persecution complex doesn't
> help to win people over to your side of things.
>
> In my experience setuptools has a lot of good ideas but they are wrapped in bad
> ideas or implementations that obscure the fact that there *are* good ideas there.
> I do not believe it to be unreasonable for people to want to make sure that we're
> standardizing around one of the *good* ideas instead of one of the bad ideas.
>
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
>

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov