[Distutils] PEP 345: real-world examples of "virtual" projects (Provides-Release)

[Sridhar Ratnakumar]

Marius Gedminas wrote:
> On Tue, Dec 21, 2010 at 01:41:55PM -0800, Sridhar Ratnakumar wrote:
>> >  Fromhttps://bitbucket.org/ametaireau/python-peps/src/tip/pep-0345.txt
>> >
>> >  """
>> >  Provides-Release (multiple use)
>> >  :::::::::::::::::::::::::::::::
>> >  [...]
>> >  A release may also provide a "virtual" project name, which does
>> >  not correspond to any separately-distributed project:  such a name
>> >  might be used to indicate an abstract capability which could be supplied
>> >  by one of multiple projects.  E.g., multiple projects might supply
>> >  RDBMS bindings for use by a given ORM:  each project might declare
>> >  that it provides ``ORM-bindings``, allowing other projects to depend
>> >  only on having at most one of them installed.
>> >  """
>
> Perhaps it was supposed to say "depend only on having at*least*  one of
> them installed"?
> That would be how Debian's virtual packages work, for example.

 From http://www.debian.org/doc/FAQ/ch-pkg_basics.en.html#s-virtual
"A [Debian] virtual package is a generic name that applies to any one of 
a group of packages, all of which provide similar basic functionality. 
For example, both the tin and trn programs are news readers, and should 
therefore satisfy any dependency of a program that required a news 
reader on a system, in order to work or to be useful. They are therefore 
both said to provide the "virtual package" called news-reader."

Is there a real-world example (in Python packaging) requiring such a 
constraint?

...

To me, only "at most one of them installed" makes sense ... and *only* 
for binary distributions (except 'setuptools' being provided by 
Distribute), viz.

[quote]
4. Personally, I have needs for "virtual" packages from a binary (not 
source) distribution perspective. For example, "MySQL-python" can be a 
virtual package "provided by" the binary distributions: mysql5.1-python, 
mysql5.0-python; similarly, "psycopg2" can be provided by psycopg2-pg8.4 
and psycopg2-pg9.0. Or, "modwsgi" can be provided by modwsgi-apache2.2, 
modwsgi-apache2.4. How would PEP 345's "Provides-Release" help, if at 
all, in describing this scenario?
[endquote]

Just to provide a context, I'd like to add support for virtual packages 
in PyPM[1] to facilitate providing such binary variants as above for 
some Python packages. And I'd like to understand and stick to the 
standards[2] as much as possible (instead of designing a proprietary 
metadata extension).

...

This is how I understand the PEP:

   $ cat psycopg2-pg9.0/DIST-INFO  # binary
   [...]
   Version: 2.3.1
   Provides-Release: psycopg2

   $ cat psycopg2/DIST-INFO  # source
   [...]
   Version: 2.2.9
   Provides-Release: psycopg2

   $ cat pgbrowser/DIST-INFO # source
   Requires-Release: psycopg2 (>=2.3)

This is what I asked in regards to installing such a virtual package:

[quote]
2. What is the expected behavior of 'pip install ORM-bindings'? To pick 
*one* of the many projects "providing" ORM-bindings?
[endquote] (emphasis added)

How is 'pip install pgbrowser'[3] expected resolve the `psycopg2' 
dependency (it may not know yet that it is a virtual package)? I have an 
idea as to how to do this in binary package repositories (such as those 
of PyPM):

   1. Index the `Provides-Release` field of all packages
   2. If `Provides-Release` is missing, use the name of the distribution
      itself
   3. Resolve dependencies (and requests from user) by looking
      at this `Provides` index, not the package *name* index.

For PyPI - step 3. will require, at the least, Simple Index to expose 
"provides" in URLs, eg:

   pypi.python.org/simple/@P/psycopg2
   pypi.python.org/simple/@P/psycopg2/psycopg2-pg9.0-2.3.1.win32.egg
   pypi.python.org/simple/@P/psycopg2/psycopg2-2.2.9.tar.gz

So `pip install pgbrowser` will have to resolve the dependency of 
psycopg2 by looking at /simple/@P/psycopg2, but *not* /simple/psycopg2.

Further, how would pip know that "psycopg2-pg9.0-2.3.1.win32.egg" is 
indeed the appropriate platform binary? The `Supported-Platform` field 
would be useless to package installers as the PEP does not specify its 
semantics (I wonder why it exists in first place).

Did I misunderstand anything? Has anyone thought of all this in regards 
to PEP 345?

-srid

[1] http://code.activestate.com/pypm/
[2] I just incorporated "environment markers" support.
[3] I use `pip` only as an example; substitute it with any package installer