[Python-Dev] Distutils and Distribute roadmap (and some words on Virtualenv, Pip)

Thu Oct 8 19:05:09 CEST 2009

Toshio Kuratomi a écrit :
> On Thu, Oct 08, 2009 at 01:27:57PM +0200, M.-A. Lemburg wrote:
>>> Tarek Ziadé a écrit :
>>>> But if PEP 376 and PEP 386 support are added in Python, we're not far
>>>> from being able to provide multiple version support with
>>>> the help of importlib.
>> Before putting much work into this: do you really think that having
>> multiple versions of the same package in the same Python installation
>> is a good idea ?
>>
> I think it is a good idea.
> 
>> Examples:
>> What if you have an application that uses two modules which each
>> require two different versions of the same package ? Would that
>> load the same package twice, causing globals used by these package
>> to no work (e.g. isinstance(x, class_name) or global caches) ?
>>
> That's not how it should work.  Look at other systems that allow for
> installing multiple versions of a library -- for instance, loading dynamic
> shared objects in C
> * You can install multiple versions of a library in parallel
> * The dynamic loader will pick the version of the library that is
>   appropriate from the list of available options (the program specifies the
>   SONAME it needs -- library name plus API version.  The loader then
>   chooses the most recent revision that matches that API version.)
> * When one binary needs multiple API versions of the library, the
>   application cannot start.
> 
> The last point addresses your concern -- depending on multiple, incompatible
> versions of a library is prohibited.  The programmer of the application
> needs to make the code run with a single version of the code.
> 
>> This sounds a lot like DLL- or RPM-hell to me.
>>
> RPM-hell (I'm not sure if DLL hell is the same, I have the vague impression
> that it is the lack of enough version specification rather than too much but
> I don't know for sure). is similar but it takes place on the end-user's
> system.  This should take place on the programmer's system instead.
> End-users are not in a position to fix things like RPM-hell.  Programmers
> are.
> 
> Example RPM-hell:
> Application Foo requires libbar-1.x
> Application Baz requires libbar-2.x
> 
> The user may either have Foo or Baz installed on their system with the
> appropriate libbar but not both.  They depend on the packagers and
> developers of Foo and Bar to do one of the following to resolve the
> situation:
> 
> * Port Foo and Baz to use the same version of libbar.
> * Package libbar in such a way that libbar-1 and libbar-2 are parallel
>   installable on the system.  Then they can install two separate packages,
>   libbar1-1.0 and libbar2-2.0.
> 
> Example of similar Distutils multiple version problem:
> The programmer creates an application Foo that depends on python-bar-1.x. He
> has recently started work on a file that imports python-baz-1.0.  python-baz
> depends on python-bar-2.x.  The first time he tries to run his new code,
> python gives him an error message that it is impossible to satisfy the
> version requirements for python-bar.  Depending on how the versions are
> specified, the error message could be very specific and helpful:
> 
>   Impossible version requirements:
>     bar Requires: python-baz>=2.0, < 3.0
>     foo.py Requires: python-baz >=1.0, < 2.0
> 
> The programmer can then discard their new code, port foo.py to
> python-baz-2.x, or port python-bar to python-baz-1.x and submit a patch to
> the upstream of that module.  Note two things about this scenario:
> 
> 1) The programmer is the person who is responsible for creating the conflict
> and for resolving it.  They are the proper authority for making the decision
> to port to python-baz-2.x or not using python-bar.  The end-user who is not
> responsible is not impacted by this at all.
> 2) The programmer would have had to deal with this issue whether we allow
> multiple versions to be installed or not.  With multiple version support we
> may be able to get them better error messages (depending on how the
> dependency information is formatted and how completely it was specified in
> the app and modules).
> 
>> I think it's much better to keep things simple and under user
>> control, e.g. by encouraging use of virtualenv-like setups
>> and perhaps adding better native support for these to Python.
>>
>> If the user finds that there's a version conflict this can
>> then be resolved during environment setup instead of hoping
>> for the best and waiting for application failures at run-time.
>>
> For the class of user that is actually a developer, it might be somewhat
> true that version conflicts should be resolved by them.  But for the class
> of user that is an end-user, version conflicts are a totally foreign
> concept.  They should be dealt with by the person who is coding the
> application for them.
> 
> Also note, the ability to have multiple versions makes things easier for
> system packagers and provides an easy alternative to a virtualenv for
> end-users.
> 
> * System packagers: virtualenv does not provide a method suitable for system

Yes, there is no doubt virtualenv is useless for system packagers but:

* System and applications deployment have not to be tied.
It s up to the user to install things system wide or to use locally isolation
technics. Virtualenv is one of those.
As a conclusion, there are not very much problem for system packagers as if
their users have specific needs, they will do something Outside the system.
If not, they use their global python with packages installed in that global one.

>   packagers.  The nearest adaptation would be for the system packager to
>   install python packages into their own hierarchy not in the PYTHONPATH.
>   Then they would need to create a virtualenv-like directory that symlinks
>   to the packages in that directory.  Then they would need to write a
>   wrapper script for the application that put that virtualenv-like directory
>   into the PYTHONPATH before any other site package directories and have
>   that wrapper call the real binary.  This is needless complication for the
>   typical virtualenv install so the work is not likely to be done there and
>   it's incredibly hacky for the system packager so the work is not likely to
>   be done there.
> * End users: virtualenv creates a whole environment for the application to
>   live in.  If python understood how to use multiple versions then we'd only
>   need to install the versions of packages that didn't match up with what's
>   already on the system into the user's personal site-package directory.
>   Then the module loader would take care of loading the module from the
>   personal site-packages since it met the version requirements instead of
>   the system copy.
> 

I dont want to have interaction between my different hosted projects, isolation
is the only thing i want. As many other people there already.
I do not want at all to have to deal with multiple version picking system and
debug an application at 2:00am because of that.

> -Toshio
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/kiorky%40cryptelium.net

-- 
--
Cordialement,
KiOrKY
GPG Key FingerPrint: 0x1A1194B7681112AF

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/78426e48/attachment-0002.pgp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 261 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20091008/78426e48/attachment-0003.pgp>