[Distutils] Low Level API for translating distutils/setuptools metatdata to Debian metadata

[Brian Sutherland]

On Wed, Jul 22, 2009 at 08:41:38AM -0600, Zooko Wilcox-O'Hearn wrote:
>>     * Provides a mapping between python project names and Debian
>>       binary/source package names
>>     * Converts setuptools versions to Debian versions while  
>> maintaining
>>       sort order
>>     * Can introspect an .egg-info directory to figure out the Debian
>>       dependencies for use in the debian/control file. It can also
>>       handle/understand extras (I Hope!)
>
> I looked briefly at this code, and it appears that it is doing purely  
> syntactic mapping between Debian package names and Python distribution 
> names, for example:
>
> def py_to_bin(setuptools_project):
>     """Convert a setuptools project name to a debian binary package  
> name"""
>     return _PY_TO_BIN.get(setuptools_project, 'python-%s' %  
> setuptools_project.lower())
>
> This works most of the time, but it isn't reliable.  For example, the  
> module name is "OpenSSL", the distribution name is "pyOpenSSL", and the 
> Debian package name is "python-openssl":
>
> http://packages.debian.org/sid/amd64/python-openssl/filelist
>
> That's why I contributed a patch to stdeb which uses the Debian database 
> of which files are included in which packages (the same database that 
> generates the web page linked above):
>
> http://github.com/astraw/stdeb/blob/647dd441a1712f8df37b5f7f5ba22ab6aeb2c3e7/stdeb/util.py#L135
>
> The way stdeb does it looks in the database for files named  
> "$DISTRIBUTION-$VERSION-py$PYTHONVERSION.egg-info".  The Debian package 
> that includes such a file is the Debian package that you need to install 
> in order to satisfy a dependency on $DISTRIBUTION, $VERSION.  This works 
> regardless of whether the Python package is built with distutils or 
> setuptools, and indeed it works for all packages that I know of.  (There 
> is actually one exception: the Debian package for setuptools itself 
> doesn't include a version number in its .egg-info filename:
>
> http://packages.debian.org/sid/amd64/python-setuptools/filelist
>
> It has a file named:
>
> /usr/share/pyshared/setuptools.egg-info/
>
> I guess we should add a fall-back-with-warning behavior to stdeb that if 
> it can't find "$DISTRIBUTION-$VERSION.egg-info", but it can find  
> "$DISTRIBUTION.egg-info", then it should (optionally) assume that the  
> Debian package that has that file will satisfy the requirement,  
> regardless of the version number in the requirement.   That, or someone 
> should open a ticket asking Debian to add a version number to that 
> filename.
>
> The exact regexp is currently:
>
> egginfore=("(/(%s)(?:-[^/]+)?(?:-py[0-9]\.[0-9.]+)?\.egg-info)"
>                % '|'.join(req.project_name for req in requirements))
>
> If you would be interested in including this mechanism to query the  
> database in van.pydeb, I would be happy to advise you.

Hi Zooko,

van.pydeb is designed to be run at package build time, rather than at
the time you create the source package. I assume that's when stdeb's
code runs? Using apt-file during package build time on one of the Debian
project's auto-builders will not be acceptable (I assume).

Also, "isn't reliable" has very different meanings :) van.pydeb is
reliable in that it produces the same answer independently of the
machine it's run on. But it's not reliable in that often the answer is
just plain wrong;)

The results of stdeb's mechanism depend on the configuration of the
machine where you run it (or even if you havn't run apt-file update
recently).

So, I don't think van.pydeb can use stdeb's mechanism as-is, but I'd
love to see a patch (with tests!) for a variation of it. I was thinking
of writing a script/function that could use apt-file to generate a list
of python->debian package mappings that don't fit the heuristic.

So py_to_bin could be re-written as:

    def py_to_bin(setuptools_project):
        """Convert a setuptools project name to a debian binary package name"""
        result = _HANDWRITTEN_PY_TO_BIN.get(setuptools_project)
        if result is None:
            result = _APT_FILE_GENERATED_PY_TO_BIN.get(setuptools_project)
        if result is None:
            result = 'python-%s' % setuptools_project.lower())
        return result

The _APT_FILE_GENERATED_PY_TO_BIN could be re-generated periodically.
I'm hoping that'll be enough given that:

    * packages don't change names that frequently
    * there should be few that don't fit the heuristic (*cough* policy *cough*)

> Regards,
>
> Zooko

-- 
Brian Sutherland