[Distutils] formencode as .egg in Debian ??

[Phillip J. Eby]

At 07:54 AM 11/25/2005 +1100, David Arnold wrote:
> From a Python-centric viewpoint, Debian's (and RedHat, Gentoo, Solaris,
>etc) packaging mechanism, however great, covers only one of the many
>possible platforms that an application might need to support.
>
> From a Debian-centric viewpoint, dependency management is a solved
>problem, with a decade of experience behind it, and proposing that
>Python applications should have their dependencies managed separately is
>blatantly foolhardy.

I prefer a slightly different spin:

Python developers would *love* to have Debian manage their packages, they 
would simply like to be able to verify at runtime that the management has 
in fact been done.  It's not that we don't trust you, it's just that we're 
paranoid.  :)  We'd like for Debian to advertise to our packages, precisely 
what versions of which of our dependencies are installed.  We'd also like 
for Debian to include the metadata we provided with our packages, when it 
installs them.  And we'd like all this to cleanly work with any 
locally-installed non-Debian eggs that might be in the mix, since we need 
to do development, beta testing, etc.


>I'm not convinced that's the case (I'm not sure how reasonable it is to
>automatically convert 'require("FooBar>=1.2")' into the equivalent
>Debian package names and versions, but my expectation is that it's no
>more difficult than it currently is to package a "product" for Debian.

The 'FooBar' name generally refers to a well-known name registered with 
PyPI, and both the name and version come from metadata that is already 
supplied by setup.py for all distributions packaged with the distutils 
today.  So you actually have this information *already*, for any 
distutils-based distribution you package with Debian.

For the Python developer's purposes (i.e., verifying whether something is 
there), you do not *need* to match this up with Debian package naming, 
however.  You only need to do that when you are packaging something that 
uses such dependencies, and as you say, it's probably not any harder than 
the dependency mapping you do now.  (Except for the part where egg-based 
projects tend to have more dependencies simply because they *can*, 
encouraging reuse instead of reinvention.)

Anyway, for advertising the presence of dependencies, we only need a 
standardized way for distributors to say, "FooBar 1.2 is installed 
here".  If a developer's project needs FooBar 1.2, he's not really worried 
about what Debian package provides it, and the egg runtime isn't in the 
business of asking system packagers to install things.  It just "properly 
complains" (as one person put it) about the absence of things that should 
be there.


>Finally, this is ignoring other possible uses of egg metadata, which
>might end up being far more problematic?

I don't see why they would be any more problematic than any other kind of 
data that comes with Python packages.  Again, we just need to be able to 
read this information at runtime to discover services, plugins, etc. 
without requiring user configuration besides just installing things.

Here's a motivating example of what this kind of data can be used 
for.  Suppose I said, "okay, I don't see how we can satisfy the Debian 
requirements cleanly, so suppose I make setuptools allow plugins to 
redefine how the metadata gets found?"

I would then define an entry point group, let's say 
"setuptools.dependency_resolvers", and add some code to query these entry 
points when scanning for egg metadata.  Debian could then create a separate 
Python package, and include in its metadata an entry point for this entry 
point group, which would tell setuptools where to import the plugin code 
from.  Debian would then make sure this package and its metadata are 
discoverable to setuptools, and setuptools would then automatically do the 
right thing on any Debian installation, even if the user was running with a 
new, locally-installed version of setuptools.

Of course, this scenario wouldn't actually work due to the bootstrapping 
issue.  I just mention it so you can see an example of why packages want to 
be able to discover metadata besides the mere presence of a dependency.  In 
this example, the hypothetical Debian-written plugin need not be known 
about by the provider of the plugin point, i.e. setuptools.

Anyway, the point is that we simply need a way to link that kind of data to 
sys.path entries, so we can be sure that the things we need are there, and 
not shadowed by things that are before them on sys.path.