[Distutils] SOLVED: bdist_rpm and pre-release python packages / eggs (was: pre-release versioning problems with sdist, bdist_rpm, bdist_debian)

Manuel Amador (Rudd-O) rudd-o at rudd-o.com
Fri Mar 13 19:18:07 CET 2009


Bill,

Thank you very much for sharing your code with us.  I am pretty sure it works 
quite well because it's more-or-less what RPM does (bar general alphabetical 
sort of the non-numeric parts of the release).  It'd be pretty great if this 
code had been the foundation for distutils from the start.  

Regrettably, we need to cope with the few thousand eggs that already follow 
the existing naming convention based on the Python setuptools implementation, 
and the widespread deployment of existing setuptools implementations as well.  
That makes it pretty clear that ascribing the cause of the problem to "bad 
training only" is misleading at best.

Dynamically rewriting the version numbers at RPM biuld time -- exclusively for 
pre-release packages -- sort of patches over the issue with fairly good 
results and no need to change a single line of deployed code, a single line of 
existing eggs, the need for retraining or cumbersome per-package intervention.

Now, the process isn't complete.  We already have the correct version 
rewriting technology, and on top of that the auto-dependency technology as 
well, but there are a few loose ends that we still need to tie, and three 
things come to my mind now:

1.

Pure python modules should be built to install in site-python, so they can be 
imported everywhere (regardless of architecture, available python interpreter 
version or distribution) without N*M rebuilds.  I already volunteered to write 
this, which amounts to a two-liner patch.

2.

(this is only Fedora-specific and I think it's fixed already) bdist_rpm should 
auto-create the .pyo files so they have selinux security contexts.

3.

- the python compilation miniscript that bdist_rpm uses to install .pyc files 
fails in a number of packages that contain pseudo-python files (files that 
have the extension .py but do not really contain python code -- this is 
prevalent in Plone products) -- the INSTALLED_FILES contains the list of py 
and pyc files but does not take into account the failure to build some pyc 
files, thus the RPM build process bombs out because it does not find the auto-
specified files at all.

In my Plone build tree I have worked around all of these issues successfully 
by using a custom RPM %install shellscript, so I have real confidence that I 
can port these shellscripts directly into distutils (they should be at most a 
three four-liner patches, and of course the fedora-specific fix does not need 
to go into distutils at all).

Oh, and the good news is, I have built Zope and Plone 3.2.2 for (at least) 
CentOS (and other RPM distros should just work, if not, msg me privately to 
report it as a bug and I will fix it).  They are at:

http://yum.rudd-o.com/

together with the scripted build process available there too.

And thanks for breathing some common sense in this discussion.

El Viernes 13 Marzo 2009, Bill Campbell escribió:
> On Fri, Mar 13, 2009, Gerry Reno wrote:
> > Manuel,
> >  'bdist_rpm' is NOT broken.  What is broken is packagers misuse of the
> > 'version' and 'release' strings.  They do stupid things like put
> > version='3.0' and release='rc1' and then wonder why their final release
> > cannot update the release candidate.  THIS IS A TRAINING ISSUE...
>
> This is a pretty good description of the cause of the problem,
> but doesn't address a solution that may be used in software.
>
> I deal mostly with building RPMs used under the OpenPKG portable
> package management system which generally uses sane naming
> conventions, at least as far as the release designations while
> versions depend on the original package versions.
>
> The attached bit of code is basically what we use to determine
> the most recent packages, and works by splitting the version and
> release into tuples of numeric and non-numeric parts, then
> comparing these tuples.  This is based on a recipe in the
> O'Reilly Python Cookbook to sort file names containing numerics.
>
> This seems to solve most of the problems of sorting RPM packages
> on version and release, but might fail in places where there are
> issues of case-sensitivity.
>
> Bill


-- 

	Manuel Amador (Rudd-O) <rudd-o at rudd-o.com>
	Rudd-O.com - http://rudd-o.com/
	GPG key ID 0xC8D28B92 at http://wwwkeys.pgp.net/

Women are more easily and more deeply terrified ... generating more
sheer horror than the male of the species.
		-- Spock, "Wolf in the Fold", stardate 3615.4

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090313/0d4309a9/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20090313/0d4309a9/attachment.pgp>


More information about the Distutils-SIG mailing list