bdist_deb (was: Extending distutils with 3rd party build commands?)

On Fri Oct 22 00:42:47 CEST 2004 Mark W. Alexander wrote:
I'm chomping to do a bdist_deb, but sitting on my hands hoping someone who can contribute it will do it first.
I've got one that pretty-much works. I'd be happy to get rid of the last couple peculiarities that I know of and massage it into a patch on CVS as long as someone assures me that there's a decent chance the patch will be accepted. Best Regards, Jeff Dairiki

Geoffrey T. Dairiki wrote:
That would be great. The only thing to remember is that the patch will have to be Python 2.1 compatible. Do we have Debian folks in the Python-Dev team that could review the patch from a Debian point of view ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 25 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
Regards, Bastian - -- ,''`. Bastian Kleineidam . calvin (at) debian.org : :' : `. `' GnuPG Schlüssel http://kampfwurst.net/gpgkey.txt `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBfNJMeBwlBDLsbz4RAreEAKCq+F8BdLP1jUEt3sRXg55qxlOi+gCeMVn3 mpoOBDWf2NA7ynXteLtPL3E= =dw3Q -----END PGP SIGNATURE-----

On Mon, 25 Oct 2004 12:15:41 +0200, Bastian Kleineidam <calvin@users.sourceforge.net> wrote:
It would be great if you could. The patch set is available from SourceForge's patch tracker for the Python project: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 Best Regards, Jeff

It looks like some good progress has been made on this front since the original posting. Are we at a point where it's ready to be checked in now? Thoughts? Thanks, Sean -- moshez is not afraid of commitment. "Yes, honey, I'm ready for commitment. Why, I commit to numerous CVS projects" Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Sat, 15 Jan 2005 17:38:54 -0700, Sean Reifschneider <jafo@tummy.com> wrote:
I (the author of the patch) say it's ready. On the other hand, I haven't really heard a lot of feedback on it from, e.g., higher ups in the debian-python world, so you may or may not want to take my opinion with a grain of salt. Jeff

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Mon, Jan 17, 2005 at 03:18:59PM -0800, Jeff Dairiki wrote:
- - The remove_tree method in bdist_deb is also available in dir_utils.py. I think it can be replaced with from distutils.dir_utils import remove_tree - - The bdist_deb.in_directory() utility function should probably be moved to spawn.py so that other commands can use it too. - - There is not a way to supply additional debuild options. My current debuild options are used for signing the package (-pgpg -sgpg -k$(GPGKEY)), and running automated package checks linda and lintian (--linda --lintian-opts -i --linda-opts -i). It might not be a bad idea to supply a generic --debuild-opts and --dpkg-buildpackage-opts since I think most developers use different options for debuild. - - The dh_make command puts (in my case) the complete GPL text in debian/copyright. To reduce disk usage the preferred way is to mention only the name and refer to the complete text in /usr/share/common-licenses, ie. License: GPL , see /usr/share/common-licenses/GPL for more info. Currently available entries in common-licenses directory: - -rw-r--r-- 1 root 6111 Dec 16 1996 Artistic - -rw-r--r-- 1 root 1499 Aug 26 1999 BSD lrwxrwxrwx 1 root 5 Dec 17 14:04 GPL -> GPL-2 - -rw-r--r-- 1 root 17992 Sep 16 1999 GPL-2 lrwxrwxrwx 1 root 8 Dec 17 14:04 LGPL -> LGPL-2.1 - -rw-r--r-- 1 root 25284 Feb 2 2000 LGPL-2 - -rw-r--r-- 1 root 26528 Jul 21 2003 LGPL-2.1 - - The generated debian/rules executes dh_python. This needs a build dependency on python which is currently not there. - - When running debuild standalone (after python setup.py dh_make), generating the PKG-INFO file fails in the case when setup.py imports additional modules at the top. Here is the error I get: $ debuild binary generating debian/PKG-INFO Traceback (most recent call last): File "<string>", line 1, in ? File "/usr/local/lib/python2.5/distutils/core.py", line 218, in run_setup execfile(script_name, g, l) File "setup.py", line 292, in ? include_dirs = include_dirs + \ File "setup.py", line 53, in normpath return os.path.normpath(path) NameError: global name 'os' is not defined make: *** [debian/PKG-INFO] Fehler 1 The 'os' module is imported in setup.py at the top. - - After running python setup.py dh_make there should be a note reminding the packager that he has to review and edit the generated files in the debian/ subdirectory. - - Manpage stubs debian/*.1 for the scripts should only be generated when such a file is not already installed. For example the setup.py from LinkChecker already installs manpages when running on POSIX platforms: [...] data_files.append(('share/man/man1', ['doc/en/linkchecker.1'])) data_files.append(('share/man/de/man1', ['doc/de/linkchecker.1'])) data_files.append(('share/man/fr/man1', ['doc/fr/linkchecker.1'])) [...] - - Both commands dh_make and bdist_deb should probably not be run on non-POSIX platforms. This should be documented. If you run it on eg. a Windows platform, an appropriate error message should be printed, something like this: if os.name != 'posix': print "This command is only available on POSIX systems" print "Your current system is %s (%s)." % (os.name, sys.platform) sys.exit(1) - - dh_make should also support generating packages relying on the "default" Python version (according to section 3.1 of the Debian Python Policy). This means there is a single package name (without the "python-" prefix) depending on python (>= 2.3), python (<< 2.4). Well, that's it for now :) Regards, Bastian - -- ,''`. Bastian Kleineidam : :' : GnuPG Schlssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFB7Pl/eBwlBDLsbz4RAhFZAKCLbxrNrjFGqdC5Ad+vB/7vB0V0QACfff+G Bd/dWB8RPrmpSURUAEktLCk= =Ae8k -----END PGP SIGNATURE-----

On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
If nobody else is available I can do that, but I'm not part of python-dev, and have a very busy schedule during the next two weeks. You may want to ask some feedback on debian-python@debian.org too. -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org

Marc-Andre wrote:
The only thing to remember is that the patch will have to be Python 2.1 compatible.
Comments in the source code for, e.g. bdist_rpm suggest that things should remain compatible with Python 1.5. Is that right, or is 1.5 no longer supported? Alexandre wrote:
You may want to ask some feedback on debian-python@debian.org too.
Okay. I'll get the patch together and posted (hopefully by sometime tomorrow), so that it's available for perusal, then I'll post a note on debian-python. I've been pretty careful about reading and obeying debian-policy and python-policy, and have been running the bdist_deb generated packages through lintian for testing. Thanks for the comments! Jeff

On Monday 25 October 2004 23:03, Jeff Dairiki wrote:
http://mail.python.org/pipermail/distutils-sig/2004-September/004168.html -- -jeff

On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
Do we have Debian folks in the Python-Dev team that could review the patch from a Debian point of view ?
I can look at it. Please copy me directly on the patch and/or put it in the Python sourceforge tracker assigned to me, I only check this list infrequently. Thanks, Sean -- Thieves broke into Scotland Yard yesterday and stole all the toilets. Detectives say they have nothing to go on. Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Mon, Oct 25, 2004 at 07:23:04PM -0600, Sean Reifschneider wrote:
Put it in the tracker and I'll steal^W take a look too. I take it you're looking for Debian Policy conformance, or is something else concerning you? If the former, I believe that's only a concern for "official" packages, although the closer you get the better. mwa -- Mark W. Alexander slash@dotnetslash.net The contents of this message authored by Mark W. Alexander are released under the Creative Commons Attribution-NonCommercial license. Copyright of quoted materials, if any, are retained by the original author(s). http://creativecommons.org/licenses/by-nc/2.0/

Mark W. Alexander wrote:
I don't "live" in the Debian space, but since every community has its standards I would just like to see the generated Debian packages conform to these standards. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 26 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

Marc-Andre, Mark, Sean, I've posted my bdist_deb patches to the SF tracker: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 There's a slightly detailed README.bdist_deb included within the patch. I'm open to all comments. Thanks for taking a look! Jeff Dairiki

I'm just doing a review of this code. A couple of things: There's been some concern expressed about get_default_maintainer. Namely, that if debchange changes it's algorithm, it won't be reflected in this code. It seems like one possible way around that would be to build a directory with a "debian" directory under it, a fake "changelog", and then call debchange to write the data out, and parse it. Too bad there's not a direct hook into debchange to get that information. Can _formatdate, if email doesn't exist, use rfc822.formatdate()? Ditto for _parseaddr? It looks pretty good. However, when trying to build a .deb of my jotweb2 package, it's failing with: [...] dh_testdir dh_testroot dh_installchangelogs dh_installdocs cp: cannot stat `doc': No such file or directory dh_installdocs: command returned error code 256 [...] I'm not sure exactly why. I do have a "doc" directory in my main package directory, but I don't reference to it in my setup.py or MANIFEST. Adding it to the MANIFEST doesn't seem to help this. You can try picking up the jotweb code from ftp.tummy.com:/pub/tummy/jotweb/ and pick up the 1.05 tar file. Sean -- Put out fires during the daytime. Do your real work at night. Sleep is just an addiction. -- Dieter Muller Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

Thanks for the comments, Sean. On Tue, 2004-10-26 at 20:48, Sean Reifschneider wrote:
That seems like a lot of work. Is it really worth it? A (similar) alternative would be to use debchange to generate the real debian/changelog. Since debchange fails to work unless there's already a non-empty changelog, that entails creating an initial changelog with a dummy entry, running debchange, and then cleaning up the dummy entry --- also kind of messy.
Can _formatdate, if email doesn't exist, use rfc822.formatdate()?
rfc822.formatdate is broken in my (Debian's) python2.1 and python2.2 in that it uses strftime (and thus day and month names might come out in a language other than English). Also they do not include a numeric timezone designation. (They use 'GMT' rather than '-0000'). Debian policy specifies that the zone designation "should be" numeric. I'm not sure whether it causes problems if it's not, but I suspect it might. (I've just noticed that dpkg-dev includes an '822-date' program, which could be used in lieu of formatdate, I guess.)
Ditto for _parseaddr?
Yes, rfc822.parseaddr will work. Though now you've prompted me to take a look at how debchange parses email addresses, and it's considerably dumber than either rfc822.parseaddr or email.Utils.parseaddr. To wit: ... $env{'DEBEMAIL'} =~ /^(.*)\s+<(.*)>$/ ... Using debchange to generate the changelog entry would eliminate the need for parseaddr and formatdate altogether, so perhaps that is the best way to go. Comments?
It looks like you don't use sdist to build your source distribution, since it includes files which aren't in the MANIFEST. (What's happening is that bdist_deb sees the 'doc' directory and adds it to the default value for it's --doc-files option, but the sdist command doesn't include the doc subdirectory in the source distribution. You would have to add at least one file from the doc subdirectory to MANIFEST to fix this, I think. A better solution might be to add 'graft doc' to MANIFEST.in. You could also, of course, explicitly set the --doc-files option, e.g. 'python setup.py bdist_deb --doc-files=README'.) Overnight, I've come to believe that bdist_deb is being overly aggressive about guessing default values for --doc-files. I guess, at the least, that I need to make sure the assumed doc files are in fact present in the source distribution. (At one point, I did try using sdist --manifest-only to generate a MANIFEST, then picking doc-files out of the MANIFEST, however this bug was causing problems with that:) http://sourceforge.net/tracker/index.php?func=detail&aid=1052827&group_id=5470&atid=105470 As far as I can tell, README and README.txt are the only doc-type files that distutils treats specially. (They automatically get added to the MANIFEST.) Are there any PEPs or other standards which suggest other standard names for doc files? Currently, to compute the default value for --doc-files, bdist_deb looks for the following files or directories: README, README.txt, CHANGES, CHANGES.txt, NEWS, NEWS.txt, CREDITS, COPYING, AUTHORS, changelog, ChangeLog, and doc After sleeping on it, my current thinking is to drop 'doc' from the list, and only look for files (not directories). After seeing jotweb2, I'm think of adding 'LICENSE' and 'LICENCE' to the list. Comments? (This turned out to be rather a longer reply than I intended. Thanks for sticking it out to the end.) Jeff

Thanks to all for their comments. I've got a second attempt at a bdist_deb ready. If put a new set of patches up at: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 I've renamed the 'debianize' command to 'dh_make', added a better test script, and it now work on woody. A more detailed changelog is included in README.bdist_deb which is included within the patch set. As always, comments would be appreciated. Jeff

That seems like a lot of work. Is it really worth it?
Probably not. I'll leave that decision up to you, and plan to bug you if the format changes such that having this would have prevented code from breaking.
rfc822.formatdate is broken in my (Debian's) python2.1 and python2.2 in that it uses strftime (and thus day and month
Ok, was just wondering. Less code means less to maintain, but if it doesn't work then use the other implementation.
That seems like a win. Adding a fake changelog entry is just a few lines of code, right? Stripping it out shouldn't be too bad I would guess, just a few more lines with a regex? Checking for errors in the run a few more? Wouldn't that make it more accurate, allow it to possibly follow changes to the Debian code without modification of the Python code, and get rid of a bunch of other code?
It looks like you don't use sdist to build your source distribution, since it includes files which aren't in the MANIFEST.
That's true. I have my own process for building the source tar which involves CVS, tags, a clean checkout of that tag from the repository, and running a script if present. Since not all my code releases are Python, I'd prefer if I could continue releasing code in that way, but would consider another option if required.
file from the doc subdirectory to MANIFEST to fix this, I think. A better solution might be to add 'graft doc' to MANIFEST.in.
Doing this has resolved that problem. Thanks. Looks like it builds fine.
In my case, I don't have README in doc, I have documentation for the system in there. README is in the top directory.
That's probably a good idea. Having the license in the package is a good thing. Sean -- I think the net needs some Viagra today. It's just not performing... -- Mike Loseke, 2000 Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Mon, 2004-11-08 at 20:30, Sean Reifschneider wrote:
Yup, it's already done, thanks to your initial suggestion. The current set of patches (posted on the patch tracker) uses debchange to generate the debian.changelog.
I've also modified things so that the dh_make stage looks for doc files to include in the distribution after the sdist building stage. This avoids the error you were experiencing. (But your doc files would still not make it into the Debian package unless you included them in the MANIFEST(.in).)
Understood. My code (if you don't default the --doc-files option to bdist_deb) uses some (fairly dumb) heuristics to look for files which should be placed in the /usr/share/doc/<package-name>/ directory within the Debian binary package. Files in that directory include some Debian-specific files like the debian changelog, but also "should" include things like the python packages README, and probably things like your doc/ subdirectory. I was wondering if there are any Python-specific standards or suggestions which suggested names for these doc-like files. (The GNU coding standards, http://www.gnu.org/prep/standards/, for example specify names for certain files like NEWS, INSTALL, README, COPYING, ChangeLog.) I've looked for an appropriate PEP and not found it. The only evidence of any special filenames I've found is that distutils sdist looks for README and README.txt. Anyhow, thanks again for the comments, Sean. Try out the latest patches when you get a chance. The latest patchset (fresh yesterday) can be found at http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 Jeff

Geoffrey T. Dairiki wrote:
As always, I think explicit is better than implicit. The bdist_rpm command uses an option with which you can specify the doc-files (--doc-files). I'd use the same approach for bdist_deb. That way a packager has full control over what is considered a documentation file and what not. I usually place that information into the setup.cfg file, so that the user doesn't have to bother about the setting. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 09 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Tue, 09 Nov 2004 17:03:16 +0100, M.-A. Lemburg <mal@egenix.com> wrote:
My code current does implement a --doc-files option, a la bdist_rpm. There are also several other options which control what goes into /usr/share/doc: --upstream-changelog, --license-file, and --examples. The heuristics are only used if the option is not explicitly set (e.g. if the --doc-files=NEWS option is specified, README will not automatically be included.) My thinking is that it's better to shotgun a few things, like a README and hopefully some licensing information, into the package rather than just leave them out. The reason for the four separate options rather than just --doc-files is that Debian policy want to see those four different classes of files installed in quite specific manners. Given that the heuristics are only used to compute default values, do you still think they are a bad thing? Jeff

Jeff Dairiki wrote:
Not at all. I just got the impression that you were guessing a bit much. If it is possible to override these suggestions then that's fine. BTW, why do you need so many different options for specifying doc files ? Is that a Debian thing or requirement ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 10 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Wed, 10 Nov 2004 11:10:38 +0100, M.-A. Lemburg <mal@egenix.com> wrote:
BTW, why do you need so many different options for specifying doc files ? Is that a Debian thing or requirement ?
Yes, precisely. It's a Debian thang. The upstream changelog needs special treatment since, in the Debian package, it is supposed to be named /usr/share/doc/<package>/changelog.gz. (Not ChangeLog, CHANGES, or whatever.) Hence the --upstream-changelog option. The license file needs special treatment because it is supposed to be incorporated into the /usr/share/doc/<package>/copyright file. (The copyright file is also supposed to list the upstream author, where the upstream source can be obtained, and who the Debian packager is.) Hence --license-file. Examples are supposed to go in /usr/share/doc/<package>/examples/. Hence --examples All other documentation (except man and info pages) just (as you would naively expect) get copied into /usr/share/doc/<package>/. That's the --doc-files option. References: http://www.debian.org/doc/debian-policy/ch-docs.html In particular: http://www.debian.org/doc/debian-policy/ch-docs.html#s-changelogs http://www.debian.org/doc/debian-policy/ch-docs.html#s-copyrightfile Jeff

Geoffrey T. Dairiki wrote:
I think you should rename the "debianize" command to something like "bdist_deb_prepare" or the like. Maybe even make it a bdist_deb subcommand. The reason is that this command is likely only going to be used together with bdist_deb unless I misunderstood its purpose. Perhaps we need a new class of commands like "aux_deb", "aux_rpm", etc. that add auxiliary directories and files to a package ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 27 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Wed, 2004-10-27 at 01:26, M.-A. Lemburg wrote:
What if I rename 'debianize' to 'dh_make'? That should look attractive only to Debian hackers --- and should instantly convey the commands purpose to them as well.
The 'debianize' command is intended to be used _instead_ of bdist_deb. I see two separate classes of debian packers: 1. Those who just want a binary package, so that they can install it on their Debian machines, taking advantage of Debian's package control. This is the target audience for bdist_deb. 2. Those (either the package author/maintainer himself or a downstream Debian packager) who want to package the python package for inclusion in a/the Debian package repository. It will be tricky for a bdist_deb-like distutils command to make these people happy. Older entries in the debian/changelog need to be maintained (this could be arranged somehow, I guess). But also, it is likely that any automatically-generated set of debian/ files is going to need to be hand-tweaked a bit. The 'debianize' command is intended for this case. It is intended that it be run only once. Thereafter the debian subdirectory is maintained, either as part of the source distribution, or as part of the Debian source package. It is intended that after running 'debianize' (and probably hand-editing the results) the Debian packages are build the Debian way (e.g. using debuild.) Clear? Jeff

Geoffrey T. Dairiki wrote:
That would be great. The only thing to remember is that the patch will have to be Python 2.1 compatible. Do we have Debian folks in the Python-Dev team that could review the patch from a Debian point of view ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 25 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
Regards, Bastian - -- ,''`. Bastian Kleineidam . calvin (at) debian.org : :' : `. `' GnuPG Schlüssel http://kampfwurst.net/gpgkey.txt `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFBfNJMeBwlBDLsbz4RAreEAKCq+F8BdLP1jUEt3sRXg55qxlOi+gCeMVn3 mpoOBDWf2NA7ynXteLtPL3E= =dw3Q -----END PGP SIGNATURE-----

On Mon, 25 Oct 2004 12:15:41 +0200, Bastian Kleineidam <calvin@users.sourceforge.net> wrote:
It would be great if you could. The patch set is available from SourceForge's patch tracker for the Python project: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 Best Regards, Jeff

It looks like some good progress has been made on this front since the original posting. Are we at a point where it's ready to be checked in now? Thoughts? Thanks, Sean -- moshez is not afraid of commitment. "Yes, honey, I'm ready for commitment. Why, I commit to numerous CVS projects" Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Sat, 15 Jan 2005 17:38:54 -0700, Sean Reifschneider <jafo@tummy.com> wrote:
I (the author of the patch) say it's ready. On the other hand, I haven't really heard a lot of feedback on it from, e.g., higher ups in the debian-python world, so you may or may not want to take my opinion with a grain of salt. Jeff

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, On Mon, Jan 17, 2005 at 03:18:59PM -0800, Jeff Dairiki wrote:
- - The remove_tree method in bdist_deb is also available in dir_utils.py. I think it can be replaced with from distutils.dir_utils import remove_tree - - The bdist_deb.in_directory() utility function should probably be moved to spawn.py so that other commands can use it too. - - There is not a way to supply additional debuild options. My current debuild options are used for signing the package (-pgpg -sgpg -k$(GPGKEY)), and running automated package checks linda and lintian (--linda --lintian-opts -i --linda-opts -i). It might not be a bad idea to supply a generic --debuild-opts and --dpkg-buildpackage-opts since I think most developers use different options for debuild. - - The dh_make command puts (in my case) the complete GPL text in debian/copyright. To reduce disk usage the preferred way is to mention only the name and refer to the complete text in /usr/share/common-licenses, ie. License: GPL , see /usr/share/common-licenses/GPL for more info. Currently available entries in common-licenses directory: - -rw-r--r-- 1 root 6111 Dec 16 1996 Artistic - -rw-r--r-- 1 root 1499 Aug 26 1999 BSD lrwxrwxrwx 1 root 5 Dec 17 14:04 GPL -> GPL-2 - -rw-r--r-- 1 root 17992 Sep 16 1999 GPL-2 lrwxrwxrwx 1 root 8 Dec 17 14:04 LGPL -> LGPL-2.1 - -rw-r--r-- 1 root 25284 Feb 2 2000 LGPL-2 - -rw-r--r-- 1 root 26528 Jul 21 2003 LGPL-2.1 - - The generated debian/rules executes dh_python. This needs a build dependency on python which is currently not there. - - When running debuild standalone (after python setup.py dh_make), generating the PKG-INFO file fails in the case when setup.py imports additional modules at the top. Here is the error I get: $ debuild binary generating debian/PKG-INFO Traceback (most recent call last): File "<string>", line 1, in ? File "/usr/local/lib/python2.5/distutils/core.py", line 218, in run_setup execfile(script_name, g, l) File "setup.py", line 292, in ? include_dirs = include_dirs + \ File "setup.py", line 53, in normpath return os.path.normpath(path) NameError: global name 'os' is not defined make: *** [debian/PKG-INFO] Fehler 1 The 'os' module is imported in setup.py at the top. - - After running python setup.py dh_make there should be a note reminding the packager that he has to review and edit the generated files in the debian/ subdirectory. - - Manpage stubs debian/*.1 for the scripts should only be generated when such a file is not already installed. For example the setup.py from LinkChecker already installs manpages when running on POSIX platforms: [...] data_files.append(('share/man/man1', ['doc/en/linkchecker.1'])) data_files.append(('share/man/de/man1', ['doc/de/linkchecker.1'])) data_files.append(('share/man/fr/man1', ['doc/fr/linkchecker.1'])) [...] - - Both commands dh_make and bdist_deb should probably not be run on non-POSIX platforms. This should be documented. If you run it on eg. a Windows platform, an appropriate error message should be printed, something like this: if os.name != 'posix': print "This command is only available on POSIX systems" print "Your current system is %s (%s)." % (os.name, sys.platform) sys.exit(1) - - dh_make should also support generating packages relying on the "default" Python version (according to section 3.1 of the Debian Python Policy). This means there is a single package name (without the "python-" prefix) depending on python (>= 2.3), python (<< 2.4). Well, that's it for now :) Regards, Bastian - -- ,''`. Bastian Kleineidam : :' : GnuPG Schlssel `. `' gpg --keyserver wwwkeys.pgp.net --recv-keys 32EC6F3E `- -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) iD8DBQFB7Pl/eBwlBDLsbz4RAhFZAKCLbxrNrjFGqdC5Ad+vB/7vB0V0QACfff+G Bd/dWB8RPrmpSURUAEktLCk= =Ae8k -----END PGP SIGNATURE-----

On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
If nobody else is available I can do that, but I'm not part of python-dev, and have a very busy schedule during the next two weeks. You may want to ask some feedback on debian-python@debian.org too. -- Alexandre Fayolle LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org

Marc-Andre wrote:
The only thing to remember is that the patch will have to be Python 2.1 compatible.
Comments in the source code for, e.g. bdist_rpm suggest that things should remain compatible with Python 1.5. Is that right, or is 1.5 no longer supported? Alexandre wrote:
You may want to ask some feedback on debian-python@debian.org too.
Okay. I'll get the patch together and posted (hopefully by sometime tomorrow), so that it's available for perusal, then I'll post a note on debian-python. I've been pretty careful about reading and obeying debian-policy and python-policy, and have been running the bdist_deb generated packages through lintian for testing. Thanks for the comments! Jeff

On Monday 25 October 2004 23:03, Jeff Dairiki wrote:
http://mail.python.org/pipermail/distutils-sig/2004-September/004168.html -- -jeff

On Mon, Oct 25, 2004 at 10:47:41AM +0200, M.-A. Lemburg wrote:
Do we have Debian folks in the Python-Dev team that could review the patch from a Debian point of view ?
I can look at it. Please copy me directly on the patch and/or put it in the Python sourceforge tracker assigned to me, I only check this list infrequently. Thanks, Sean -- Thieves broke into Scotland Yard yesterday and stole all the toilets. Detectives say they have nothing to go on. Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Mon, Oct 25, 2004 at 07:23:04PM -0600, Sean Reifschneider wrote:
Put it in the tracker and I'll steal^W take a look too. I take it you're looking for Debian Policy conformance, or is something else concerning you? If the former, I believe that's only a concern for "official" packages, although the closer you get the better. mwa -- Mark W. Alexander slash@dotnetslash.net The contents of this message authored by Mark W. Alexander are released under the Creative Commons Attribution-NonCommercial license. Copyright of quoted materials, if any, are retained by the original author(s). http://creativecommons.org/licenses/by-nc/2.0/

Mark W. Alexander wrote:
I don't "live" in the Debian space, but since every community has its standards I would just like to see the generated Debian packages conform to these standards. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 26 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

Marc-Andre, Mark, Sean, I've posted my bdist_deb patches to the SF tracker: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 There's a slightly detailed README.bdist_deb included within the patch. I'm open to all comments. Thanks for taking a look! Jeff Dairiki

I'm just doing a review of this code. A couple of things: There's been some concern expressed about get_default_maintainer. Namely, that if debchange changes it's algorithm, it won't be reflected in this code. It seems like one possible way around that would be to build a directory with a "debian" directory under it, a fake "changelog", and then call debchange to write the data out, and parse it. Too bad there's not a direct hook into debchange to get that information. Can _formatdate, if email doesn't exist, use rfc822.formatdate()? Ditto for _parseaddr? It looks pretty good. However, when trying to build a .deb of my jotweb2 package, it's failing with: [...] dh_testdir dh_testroot dh_installchangelogs dh_installdocs cp: cannot stat `doc': No such file or directory dh_installdocs: command returned error code 256 [...] I'm not sure exactly why. I do have a "doc" directory in my main package directory, but I don't reference to it in my setup.py or MANIFEST. Adding it to the MANIFEST doesn't seem to help this. You can try picking up the jotweb code from ftp.tummy.com:/pub/tummy/jotweb/ and pick up the 1.05 tar file. Sean -- Put out fires during the daytime. Do your real work at night. Sleep is just an addiction. -- Dieter Muller Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

Thanks for the comments, Sean. On Tue, 2004-10-26 at 20:48, Sean Reifschneider wrote:
That seems like a lot of work. Is it really worth it? A (similar) alternative would be to use debchange to generate the real debian/changelog. Since debchange fails to work unless there's already a non-empty changelog, that entails creating an initial changelog with a dummy entry, running debchange, and then cleaning up the dummy entry --- also kind of messy.
Can _formatdate, if email doesn't exist, use rfc822.formatdate()?
rfc822.formatdate is broken in my (Debian's) python2.1 and python2.2 in that it uses strftime (and thus day and month names might come out in a language other than English). Also they do not include a numeric timezone designation. (They use 'GMT' rather than '-0000'). Debian policy specifies that the zone designation "should be" numeric. I'm not sure whether it causes problems if it's not, but I suspect it might. (I've just noticed that dpkg-dev includes an '822-date' program, which could be used in lieu of formatdate, I guess.)
Ditto for _parseaddr?
Yes, rfc822.parseaddr will work. Though now you've prompted me to take a look at how debchange parses email addresses, and it's considerably dumber than either rfc822.parseaddr or email.Utils.parseaddr. To wit: ... $env{'DEBEMAIL'} =~ /^(.*)\s+<(.*)>$/ ... Using debchange to generate the changelog entry would eliminate the need for parseaddr and formatdate altogether, so perhaps that is the best way to go. Comments?
It looks like you don't use sdist to build your source distribution, since it includes files which aren't in the MANIFEST. (What's happening is that bdist_deb sees the 'doc' directory and adds it to the default value for it's --doc-files option, but the sdist command doesn't include the doc subdirectory in the source distribution. You would have to add at least one file from the doc subdirectory to MANIFEST to fix this, I think. A better solution might be to add 'graft doc' to MANIFEST.in. You could also, of course, explicitly set the --doc-files option, e.g. 'python setup.py bdist_deb --doc-files=README'.) Overnight, I've come to believe that bdist_deb is being overly aggressive about guessing default values for --doc-files. I guess, at the least, that I need to make sure the assumed doc files are in fact present in the source distribution. (At one point, I did try using sdist --manifest-only to generate a MANIFEST, then picking doc-files out of the MANIFEST, however this bug was causing problems with that:) http://sourceforge.net/tracker/index.php?func=detail&aid=1052827&group_id=5470&atid=105470 As far as I can tell, README and README.txt are the only doc-type files that distutils treats specially. (They automatically get added to the MANIFEST.) Are there any PEPs or other standards which suggest other standard names for doc files? Currently, to compute the default value for --doc-files, bdist_deb looks for the following files or directories: README, README.txt, CHANGES, CHANGES.txt, NEWS, NEWS.txt, CREDITS, COPYING, AUTHORS, changelog, ChangeLog, and doc After sleeping on it, my current thinking is to drop 'doc' from the list, and only look for files (not directories). After seeing jotweb2, I'm think of adding 'LICENSE' and 'LICENCE' to the list. Comments? (This turned out to be rather a longer reply than I intended. Thanks for sticking it out to the end.) Jeff

Thanks to all for their comments. I've got a second attempt at a bdist_deb ready. If put a new set of patches up at: http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 I've renamed the 'debianize' command to 'dh_make', added a better test script, and it now work on woody. A more detailed changelog is included in README.bdist_deb which is included within the patch set. As always, comments would be appreciated. Jeff

That seems like a lot of work. Is it really worth it?
Probably not. I'll leave that decision up to you, and plan to bug you if the format changes such that having this would have prevented code from breaking.
rfc822.formatdate is broken in my (Debian's) python2.1 and python2.2 in that it uses strftime (and thus day and month
Ok, was just wondering. Less code means less to maintain, but if it doesn't work then use the other implementation.
That seems like a win. Adding a fake changelog entry is just a few lines of code, right? Stripping it out shouldn't be too bad I would guess, just a few more lines with a regex? Checking for errors in the run a few more? Wouldn't that make it more accurate, allow it to possibly follow changes to the Debian code without modification of the Python code, and get rid of a bunch of other code?
It looks like you don't use sdist to build your source distribution, since it includes files which aren't in the MANIFEST.
That's true. I have my own process for building the source tar which involves CVS, tags, a clean checkout of that tag from the repository, and running a script if present. Since not all my code releases are Python, I'd prefer if I could continue releasing code in that way, but would consider another option if required.
file from the doc subdirectory to MANIFEST to fix this, I think. A better solution might be to add 'graft doc' to MANIFEST.in.
Doing this has resolved that problem. Thanks. Looks like it builds fine.
In my case, I don't have README in doc, I have documentation for the system in there. README is in the top directory.
That's probably a good idea. Having the license in the package is a good thing. Sean -- I think the net needs some Viagra today. It's just not performing... -- Mike Loseke, 2000 Sean Reifschneider, Member of Technical Staff <jafo@tummy.com> tummy.com, ltd. - Linux Consulting since 1995. Qmail, Python, SysAdmin

On Mon, 2004-11-08 at 20:30, Sean Reifschneider wrote:
Yup, it's already done, thanks to your initial suggestion. The current set of patches (posted on the patch tracker) uses debchange to generate the debian.changelog.
I've also modified things so that the dh_make stage looks for doc files to include in the distribution after the sdist building stage. This avoids the error you were experiencing. (But your doc files would still not make it into the Debian package unless you included them in the MANIFEST(.in).)
Understood. My code (if you don't default the --doc-files option to bdist_deb) uses some (fairly dumb) heuristics to look for files which should be placed in the /usr/share/doc/<package-name>/ directory within the Debian binary package. Files in that directory include some Debian-specific files like the debian changelog, but also "should" include things like the python packages README, and probably things like your doc/ subdirectory. I was wondering if there are any Python-specific standards or suggestions which suggested names for these doc-like files. (The GNU coding standards, http://www.gnu.org/prep/standards/, for example specify names for certain files like NEWS, INSTALL, README, COPYING, ChangeLog.) I've looked for an appropriate PEP and not found it. The only evidence of any special filenames I've found is that distutils sdist looks for README and README.txt. Anyhow, thanks again for the comments, Sean. Try out the latest patches when you get a chance. The latest patchset (fresh yesterday) can be found at http://sourceforge.net/tracker/index.php?func=detail&aid=1054967&group_id=5470&atid=305470 Jeff

Geoffrey T. Dairiki wrote:
As always, I think explicit is better than implicit. The bdist_rpm command uses an option with which you can specify the doc-files (--doc-files). I'd use the same approach for bdist_deb. That way a packager has full control over what is considered a documentation file and what not. I usually place that information into the setup.cfg file, so that the user doesn't have to bother about the setting. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 09 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Tue, 09 Nov 2004 17:03:16 +0100, M.-A. Lemburg <mal@egenix.com> wrote:
My code current does implement a --doc-files option, a la bdist_rpm. There are also several other options which control what goes into /usr/share/doc: --upstream-changelog, --license-file, and --examples. The heuristics are only used if the option is not explicitly set (e.g. if the --doc-files=NEWS option is specified, README will not automatically be included.) My thinking is that it's better to shotgun a few things, like a README and hopefully some licensing information, into the package rather than just leave them out. The reason for the four separate options rather than just --doc-files is that Debian policy want to see those four different classes of files installed in quite specific manners. Given that the heuristics are only used to compute default values, do you still think they are a bad thing? Jeff

Jeff Dairiki wrote:
Not at all. I just got the impression that you were guessing a bit much. If it is possible to override these suggestions then that's fine. BTW, why do you need so many different options for specifying doc files ? Is that a Debian thing or requirement ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Nov 10 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Wed, 10 Nov 2004 11:10:38 +0100, M.-A. Lemburg <mal@egenix.com> wrote:
BTW, why do you need so many different options for specifying doc files ? Is that a Debian thing or requirement ?
Yes, precisely. It's a Debian thang. The upstream changelog needs special treatment since, in the Debian package, it is supposed to be named /usr/share/doc/<package>/changelog.gz. (Not ChangeLog, CHANGES, or whatever.) Hence the --upstream-changelog option. The license file needs special treatment because it is supposed to be incorporated into the /usr/share/doc/<package>/copyright file. (The copyright file is also supposed to list the upstream author, where the upstream source can be obtained, and who the Debian packager is.) Hence --license-file. Examples are supposed to go in /usr/share/doc/<package>/examples/. Hence --examples All other documentation (except man and info pages) just (as you would naively expect) get copied into /usr/share/doc/<package>/. That's the --doc-files option. References: http://www.debian.org/doc/debian-policy/ch-docs.html In particular: http://www.debian.org/doc/debian-policy/ch-docs.html#s-changelogs http://www.debian.org/doc/debian-policy/ch-docs.html#s-copyrightfile Jeff

Geoffrey T. Dairiki wrote:
I think you should rename the "debianize" command to something like "bdist_deb_prepare" or the like. Maybe even make it a bdist_deb subcommand. The reason is that this command is likely only going to be used together with bdist_deb unless I misunderstood its purpose. Perhaps we need a new class of commands like "aux_deb", "aux_rpm", etc. that add auxiliary directories and files to a package ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Oct 27 2004)
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

On Wed, 2004-10-27 at 01:26, M.-A. Lemburg wrote:
What if I rename 'debianize' to 'dh_make'? That should look attractive only to Debian hackers --- and should instantly convey the commands purpose to them as well.
The 'debianize' command is intended to be used _instead_ of bdist_deb. I see two separate classes of debian packers: 1. Those who just want a binary package, so that they can install it on their Debian machines, taking advantage of Debian's package control. This is the target audience for bdist_deb. 2. Those (either the package author/maintainer himself or a downstream Debian packager) who want to package the python package for inclusion in a/the Debian package repository. It will be tricky for a bdist_deb-like distutils command to make these people happy. Older entries in the debian/changelog need to be maintained (this could be arranged somehow, I guess). But also, it is likely that any automatically-generated set of debian/ files is going to need to be hand-tweaked a bit. The 'debianize' command is intended for this case. It is intended that it be run only once. Thereafter the debian subdirectory is maintained, either as part of the source distribution, or as part of the Debian source package. It is intended that after running 'debianize' (and probably hand-editing the results) the Debian packages are build the Debian way (e.g. using debuild.) Clear? Jeff
participants (8)
-
Alexandre
-
Bastian Kleineidam
-
Geoffrey T. Dairiki
-
Jeff Dairiki
-
Jeff Pitman
-
M.-A. Lemburg
-
Mark W. Alexander
-
Sean Reifschneider