Re: [Distutils] Multiple version support (default installs and plugins)
At 02:13 PM 12/22/2005 +0000, Paul Moore wrote:
On 12/22/05, M.-A. Lemburg mal@egenix.com wrote:
Ian Bicking wrote:
I'm still finding it impossible to use multiple versions of a package unless none of them show up in a .pth file (i.e., none are available without requiring), I get a VersionConflict.
This comes up every now and then when discussing the benefits of easy_install.
I'd really like to understand what the use case is for having multiple versions of a package around.
Note that sys.modules (the registry of loaded modules) does not support having multiple versions of a module loaded.
I think the point is to allow multiple versions to be installed on the system, but allow an individual program, at run time, to specify which one is to be loaded. There will never be more than one version loaded in any specific instance, but if program A requires version 1 of package X, and program B requires version 2 of package X, both can run without needing a reinstall of package X in between.
Of course, this is also possible via PYTHONPATH manipulations, or sys.path modifications in the program before doing the import. All setuptools is doing is to provide a common infrastructure for handling this.
Yes, Paul is correct; supporting multiple programs that use different versions of a module is indeed the primary use case. Also, during development and debugging it can be quite convenient to switch versions of a dependency back and forth to identify the source of a problem.
Phillip J. Eby wrote:
At 02:13 PM 12/22/2005 +0000, Paul Moore wrote:
On 12/22/05, M.-A. Lemburg mal@egenix.com wrote:
Ian Bicking wrote:
I'm still finding it impossible to use multiple versions of a package unless none of them show up in a .pth file (i.e., none are available without requiring), I get a VersionConflict.
This comes up every now and then when discussing the benefits of easy_install.
I'd really like to understand what the use case is for having multiple versions of a package around.
Note that sys.modules (the registry of loaded modules) does not support having multiple versions of a module loaded.
I think the point is to allow multiple versions to be installed on the system, but allow an individual program, at run time, to specify which one is to be loaded. There will never be more than one version loaded in any specific instance, but if program A requires version 1 of package X, and program B requires version 2 of package X, both can run without needing a reinstall of package X in between.
Of course, this is also possible via PYTHONPATH manipulations, or sys.path modifications in the program before doing the import. All setuptools is doing is to provide a common infrastructure for handling this.
Yes, Paul is correct; supporting multiple programs that use different versions of a module is indeed the primary use case. Also, during development and debugging it can be quite convenient to switch versions of a dependency back and forth to identify the source of a problem.
Ok, I can see your point.
How do you assure that only one version of the package gets loaded into the Python process ?
If you use a module A that needs version 1 and at the same time a module B that requires version 2, how will this conflict get resolved ?
If you find this feature important enough, shouldn't we try to get a solution into the standard Python import mechanism rather than playing tricks with sys.path ?
At 11:38 PM 12/22/2005 +0100, M.-A. Lemburg wrote:
How do you assure that only one version of the package gets loaded into the Python process ?
I went back in the time machine and convinced Guido to make the import system only load the first module it finds with a given name. ;-)
Sorry, couldn't resist. :)
If you use a module A that needs version 1 and at the same time a module B that requires version 2, how will this conflict get resolved ?
pkg_resources.VersionConflict is raised at the point of require()-ing something that conflicts with something already in the working set.
Note that you might not get a conflict if module A accepts versions >=1, and B accepts versions >=2, and you have version 2, and it's the default version or was requested by the __main__ program, effectively making it the default.
For example, TurboGears specifies requirements for certain modules that are also depended on by its dependencies. I don't remember what they are, but for the sake of example I'll say that TurboGears depends on a particular version of FormEncode, which is also depended on by SQLObject - which TurboGears also requires. So it's something like this:
TurboGears: FormEncode>=0.5dev SQLObject>=0.8dev
SQLObject: FormEncode>=0.4
If you run a TurboGears-requiring script, it's going to request a newer FormEncode, which will then become the default. SQLObject also gets processed for dependencies, since TurboGears requires it, but since the default (FormEncode>=0.5dev) is compatible with the SQLObject requirement (FormEncode>=0.4), life just goes on with no conflict.
There are currently a few issues with this algorithm, mainly that it decides on a "default version" too early; as soon as pkg_resources is finished importing, all "default versions" for eggs that are already importable on sys.path are established and can't be changed. In 0.7, I'll be doing some work to loosen this so that the default version of a project won't be set in stone until an explicit choice is made or until the default default version has been imported.
If you find this feature important enough, shouldn't we try to get a solution into the standard Python import mechanism rather than playing tricks with sys.path ?
"Playing tricks with sys.path" *is* the standard way to do it; just look at what Guido answers every time anybody asks about needing specific versions of things. "The application should just set up its sys.path however it needs to" is more or less his standard response. The pkg_resources module just provides you with convenient routines to do what Guido has already declared to be the One Obvious Way to do it, i.e. sys.path manipulation.
participants (2)
-
M.-A. Lemburg -
Phillip J. Eby