Hi Zooko, I just implemented what I was talking about below on the van.pydeb trunk. Running the script against unstable I get a list of 565 packages that don't match the default mapping, I've added them to van.pydeb's database. The checkin is here: http://mail.zope.org/pipermail/checkins/2009-July/036397.html If you have the time, please have a look over the list to see that it contains the packages you need correctly. At least pyOpenSSL you mention below is handled correctly. I probably will make a new release of van.pydeb in a week or so. On Wed, Jul 22, 2009 at 05:52:33PM +0200, Brian Sutherland wrote:
On Wed, Jul 22, 2009 at 08:41:38AM -0600, Zooko Wilcox-O'Hearn wrote:
* Provides a mapping between python project names and Debian binary/source package names * Converts setuptools versions to Debian versions while maintaining sort order * Can introspect an .egg-info directory to figure out the Debian dependencies for use in the debian/control file. It can also handle/understand extras (I Hope!)
I looked briefly at this code, and it appears that it is doing purely syntactic mapping between Debian package names and Python distribution names, for example:
def py_to_bin(setuptools_project): """Convert a setuptools project name to a debian binary package name""" return _PY_TO_BIN.get(setuptools_project, 'python-%s' % setuptools_project.lower())
This works most of the time, but it isn't reliable. For example, the module name is "OpenSSL", the distribution name is "pyOpenSSL", and the Debian package name is "python-openssl":
http://packages.debian.org/sid/amd64/python-openssl/filelist
That's why I contributed a patch to stdeb which uses the Debian database of which files are included in which packages (the same database that generates the web page linked above):
http://github.com/astraw/stdeb/blob/647dd441a1712f8df37b5f7f5ba22ab6aeb2c3e7...
The way stdeb does it looks in the database for files named "$DISTRIBUTION-$VERSION-py$PYTHONVERSION.egg-info". The Debian package that includes such a file is the Debian package that you need to install in order to satisfy a dependency on $DISTRIBUTION, $VERSION. This works regardless of whether the Python package is built with distutils or setuptools, and indeed it works for all packages that I know of. (There is actually one exception: the Debian package for setuptools itself doesn't include a version number in its .egg-info filename:
http://packages.debian.org/sid/amd64/python-setuptools/filelist
It has a file named:
/usr/share/pyshared/setuptools.egg-info/
I guess we should add a fall-back-with-warning behavior to stdeb that if it can't find "$DISTRIBUTION-$VERSION.egg-info", but it can find "$DISTRIBUTION.egg-info", then it should (optionally) assume that the Debian package that has that file will satisfy the requirement, regardless of the version number in the requirement. That, or someone should open a ticket asking Debian to add a version number to that filename.
The exact regexp is currently:
egginfore=("(/(%s)(?:-[^/]+)?(?:-py[0-9]\.[0-9.]+)?\.egg-info)" % '|'.join(req.project_name for req in requirements))
If you would be interested in including this mechanism to query the database in van.pydeb, I would be happy to advise you.
Hi Zooko,
van.pydeb is designed to be run at package build time, rather than at the time you create the source package. I assume that's when stdeb's code runs? Using apt-file during package build time on one of the Debian project's auto-builders will not be acceptable (I assume).
Also, "isn't reliable" has very different meanings :) van.pydeb is reliable in that it produces the same answer independently of the machine it's run on. But it's not reliable in that often the answer is just plain wrong;)
The results of stdeb's mechanism depend on the configuration of the machine where you run it (or even if you havn't run apt-file update recently).
So, I don't think van.pydeb can use stdeb's mechanism as-is, but I'd love to see a patch (with tests!) for a variation of it. I was thinking of writing a script/function that could use apt-file to generate a list of python->debian package mappings that don't fit the heuristic.
So py_to_bin could be re-written as:
def py_to_bin(setuptools_project): """Convert a setuptools project name to a debian binary package name""" result = _HANDWRITTEN_PY_TO_BIN.get(setuptools_project) if result is None: result = _APT_FILE_GENERATED_PY_TO_BIN.get(setuptools_project) if result is None: result = 'python-%s' % setuptools_project.lower()) return result
The _APT_FILE_GENERATED_PY_TO_BIN could be re-generated periodically. I'm hoping that'll be enough given that:
* packages don't change names that frequently * there should be few that don't fit the heuristic (*cough* policy *cough*)
Regards,
Zooko
-- Brian Sutherland
-- Brian Sutherland