PEP 345: real-world examples of "virtual" projects (Provides-Release)
From https://bitbucket.org/ametaireau/python-peps/src/tip/pep-0345.txt """ Provides-Release (multiple use) ::::::::::::::::::::::::::::::: [...] A release may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name might be used to indicate an abstract capability which could be supplied by one of multiple projects. E.g., multiple projects might supply RDBMS bindings for use by a given ORM: each project might declare that it provides ``ORM-bindings``, allowing other projects to depend only on having at most one of them installed. """ 1. I find this to be a strange example. If only one provided project (ORM-bindings) can be installed at any time, what if anyone wants to install more than one DB backends for ORM? 2. What is the expected behavior of 'pip install ORM-bindings'[1]? To pick one of the many projects "providing" ORM-bindings? 3. Did anyone--Alexis and Tarek, in particular--think of real-world use cases for virtual projects (and even "provides" in general) other than the Zope transaction case? If yes, what are they? 4. Personally, I have needs for "virtual" packages from a binary (not source) distribution perspective. For example, "MySQL-python" can be a virtual package "provided by" the binary distributions: mysql5.1-python, mysql5.0-python; similarly, "psycopg2" can be provided by psycopg2-pg8.4 and psycopg2-pg9.0. Or, "modwsgi" can be provided by modwsgi-apache2.2, modwsgi-apache2.4. How would PEP 345's "Provides-Release" help, if at all, in describing this scenario? -srid [1] Or substitute `pip` with any package installer (easy_install, distutils2.install, pypm, enstaller).
On Tue, Dec 21, 2010 at 01:41:55PM -0800, Sridhar Ratnakumar wrote:
From https://bitbucket.org/ametaireau/python-peps/src/tip/pep-0345.txt
""" Provides-Release (multiple use) ::::::::::::::::::::::::::::::: [...] A release may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name might be used to indicate an abstract capability which could be supplied by one of multiple projects. E.g., multiple projects might supply RDBMS bindings for use by a given ORM: each project might declare that it provides ``ORM-bindings``, allowing other projects to depend only on having at most one of them installed. """
Perhaps it was supposed to say "depend only on having at *least* one of them installed"? That would be how Debian's virtual packages work, for example. Marius Gedminas -- Programs that write programs are the happiest programs in the world. -- Andrew Hume
Marius Gedminas wrote:
On Tue, Dec 21, 2010 at 01:41:55PM -0800, Sridhar Ratnakumar wrote:
Fromhttps://bitbucket.org/ametaireau/python-peps/src/tip/pep-0345.txt
""" Provides-Release (multiple use) ::::::::::::::::::::::::::::::: [...] A release may also provide a "virtual" project name, which does not correspond to any separately-distributed project: such a name might be used to indicate an abstract capability which could be supplied by one of multiple projects. E.g., multiple projects might supply RDBMS bindings for use by a given ORM: each project might declare that it provides ``ORM-bindings``, allowing other projects to depend only on having at most one of them installed. """
Perhaps it was supposed to say "depend only on having at*least* one of them installed"? That would be how Debian's virtual packages work, for example.
From http://www.debian.org/doc/FAQ/ch-pkg_basics.en.html#s-virtual "A [Debian] virtual package is a generic name that applies to any one of a group of packages, all of which provide similar basic functionality. For example, both the tin and trn programs are news readers, and should therefore satisfy any dependency of a program that required a news reader on a system, in order to work or to be useful. They are therefore both said to provide the "virtual package" called news-reader." Is there a real-world example (in Python packaging) requiring such a constraint? ... To me, only "at most one of them installed" makes sense ... and *only* for binary distributions (except 'setuptools' being provided by Distribute), viz. [quote] 4. Personally, I have needs for "virtual" packages from a binary (not source) distribution perspective. For example, "MySQL-python" can be a virtual package "provided by" the binary distributions: mysql5.1-python, mysql5.0-python; similarly, "psycopg2" can be provided by psycopg2-pg8.4 and psycopg2-pg9.0. Or, "modwsgi" can be provided by modwsgi-apache2.2, modwsgi-apache2.4. How would PEP 345's "Provides-Release" help, if at all, in describing this scenario? [endquote] Just to provide a context, I'd like to add support for virtual packages in PyPM[1] to facilitate providing such binary variants as above for some Python packages. And I'd like to understand and stick to the standards[2] as much as possible (instead of designing a proprietary metadata extension). ... This is how I understand the PEP: $ cat psycopg2-pg9.0/DIST-INFO # binary [...] Version: 2.3.1 Provides-Release: psycopg2 $ cat psycopg2/DIST-INFO # source [...] Version: 2.2.9 Provides-Release: psycopg2 $ cat pgbrowser/DIST-INFO # source Requires-Release: psycopg2 (>=2.3) This is what I asked in regards to installing such a virtual package: [quote] 2. What is the expected behavior of 'pip install ORM-bindings'? To pick *one* of the many projects "providing" ORM-bindings? [endquote] (emphasis added) How is 'pip install pgbrowser'[3] expected resolve the `psycopg2' dependency (it may not know yet that it is a virtual package)? I have an idea as to how to do this in binary package repositories (such as those of PyPM): 1. Index the `Provides-Release` field of all packages 2. If `Provides-Release` is missing, use the name of the distribution itself 3. Resolve dependencies (and requests from user) by looking at this `Provides` index, not the package *name* index. For PyPI - step 3. will require, at the least, Simple Index to expose "provides" in URLs, eg: pypi.python.org/simple/@P/psycopg2 pypi.python.org/simple/@P/psycopg2/psycopg2-pg9.0-2.3.1.win32.egg pypi.python.org/simple/@P/psycopg2/psycopg2-2.2.9.tar.gz So `pip install pgbrowser` will have to resolve the dependency of psycopg2 by looking at /simple/@P/psycopg2, but *not* /simple/psycopg2. Further, how would pip know that "psycopg2-pg9.0-2.3.1.win32.egg" is indeed the appropriate platform binary? The `Supported-Platform` field would be useless to package installers as the PEP does not specify its semantics (I wonder why it exists in first place). Did I misunderstand anything? Has anyone thought of all this in regards to PEP 345? -srid [1] http://code.activestate.com/pypm/ [2] I just incorporated "environment markers" support. [3] I use `pip` only as an example; substitute it with any package installer
Hi Srid, On 12/21/2010 09:41 PM, Sridhar Ratnakumar wrote:
1. I find this to be a strange example. If only one provided project (ORM-bindings) can be installed at any time, what if anyone wants to install more than one DB backends for ORM?
Supposing I've understood that part the right way, it's not preventing to install more than one backend for an ORM. It provides a way to have a dependency to a project providing an ORM-binding. ORM bindings are different from ORM backends in my opinion. Thus, if you do have a better example to propose, I will be very happy to discuss it here, I find the DB-related example a bit odd too (It's probably not very likely to have something like that in a real use case).
2. What is the expected behavior of 'pip install ORM-bindings'[1]? To pick one of the many projects "providing" ORM-bindings?
That's what I have in mind, yes. But until we got a determinist algorithm, we don't have any way to select the same one on different installer systems. That kind of things should be provided by distutils2, but we did not thought about that until now.
3. Did anyone--Alexis and Tarek, in particular--think of real-world use cases for virtual projects (and even "provides" in general) other than the Zope transaction case? If yes, what are they?
"Provides-Release" lists the specific projects provided with this distribution. One case I can think about is having a distribution providing two different projects. A software can have a project for the "core" features and a project for the "default-plugins". It can then provides the two projects in only one distribution. Those can be two virtual projects 'core' and 'defaultplugins', which can be *provided* by different projects then. This allows to choose between the possible projects when resolving the dependencies. setuptools and distribute can be both provided by the same virtual project.
4. Personally, I have needs for "virtual" packages from a binary (not source) distribution perspective. For example, "MySQL-python" can be a virtual package "provided by" the binary distributions: mysql5.1-python, mysql5.0-python;
Yep, I understand what you mean. I don't think the Provides-Release is adapted to deal with those. 'MySQL-python' is *not* a virtual project, but it is a real one. In my opinion, the 'Provides-Release' field and virtual project concept are not to resolve this kind of problems: they are useful to deal with different *projects* (Distribute and Setuptools) providing the same features (packaging facilities).
similarly, "psycopg2" can be provided by psycopg2-pg8.4 and psycopg2-pg9.0. Or, "modwsgi" can be provided by modwsgi-apache2.2, modwsgi-apache2.4.
Each time, those use cases are related to version distributions, or to compiler options. If you are dealing with such things, I don't think the python package index and the current fields (PEP 345) will be useful to you. I'm not even sure that we need to have some fields to support that at this level. From what I understand, you have a "psycopg2" version for each of the postgre distributions. How are we supposed to deal with that on PyPI ? Arent those dependencies at the OS level rather than the python level ?
How would PEP 345's "Provides-Release" help, if at all, in describing this scenario?
I'm not sure it will help in this case, unfortunately. Hope this helps, Alex
participants (3)
-
Alexis Métaireau
-
Marius Gedminas
-
Sridhar Ratnakumar