pre-release versioning problems with sdist, bdist_rpm, bdist_debian
When I create a set of distro packages for a specific distro, I need to distribute both a tarball and a distro-specific package such as an RPM or a deb archive. Some people like installing things from tarballs, others like using the distro package manager. So I let them have it both ways. We can generate the tarball using 'sdist' and generate the distro-package with 'bdist_rpm', etc.
This all works great as long as we're talking final releases here: 4.9.0, 5.0.0. But, whenever you start to introduce pre-release candidate versioning, eg: 5.0.0_rc2, then things unravel.
What I have been attempting to do, and is apparently is like trying to find the holy grail, is to be able to generate both tarballs and distro-packages for both pre-release candidates and final releases, that exhibit the following behavior:
tarballs: foo-5.0.0_rc1.tar.gz (version=5.0.0_rc1); (extracts a foo-5.0.0_rc1 directory; 'sdist' requires version set like this so tarball name is unique and extract dir is unique) distro-package (pre-release): foo-5.0.0-0_rc1.noarch.rpm (version=5.0.0 and release=0_rc1) distro-package (final release): foo-5.0.0-1.noarch.rpm (version=5.0.0 and release=1); 'bdist_rpm' requires version and release set like this so final release will update the pre-release candidate package
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages. 'sdist' knows nothing about 'release' string so you are forced to put the pre-release candidate information into the 'version' string' but as soon as you do that then you break 'bdist' packaging. And to complicate matters, the world seems to have settled for making pre-release candidate version strings like '5.0.0_rc1' for many applications.
For example:
We have an application, foo-5.0.0, and we want to put out some pre-release candidates for testing, so we set the version to "5.0.0_rc1" in setup.py.
We create the source distribution with: $ python setup.py sdist which creates a source archive, foo-5.0.0_rc1.tar.gz.
We extract this archive and 'cd' into the foo-5.0.0_rc1 directory and create a built distribution with: $ python setup.py bdist_rpm which creates source and binary RPMS in the form: foo-5.0.0_rc1-1.noarch.rpm.
So we think everything is fine. Everyone installs and tests using the pre-release candidate and subsequent candidates but when you eventually get to the final release, foo-5.0.0, and build your final release RPMS, foo-5.0.0-1.noarch.rpm, you find that it will not update your last pre-release candidate RPM, foo-5.0.0_rcX-1.noarch.rpm because it is not "rpm newer".
I'm hoping there is some distutils guru that can tell me how to solve this enigma or show me how to extend distutils to accomplish what is needed.
Regards, Gerry
PS: In case anyone notices, yes, I posted a similar question yesterday on the python list without much response, but the question really belongs on this list.
On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno greno@verizon.net wrote:
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages.
If I understand your problem correclty, if sdist would simply concatenate the version string and the release string to use it as a "source version" when it starts to work, you would be able to work things out ?
Regards Tarek
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno greno@verizon.net wrote:
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages.
If I understand your problem correclty, if sdist would simply concatenate the version string and the release string to use it as a "source version" when it starts to work, you would be able to work things out ?
I don't know the internals of 'sdist' but I think if there were a way to extend 'sdist' to use 'release' as well as 'version' then that might work. I would have to test that to see.
Regards. Gerry
On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno greno@verizon.net wrote:
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno greno@verizon.net wrote:
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages.
If I understand your problem correclty, if sdist would simply concatenate the version string and the release string to use it as a "source version" when it starts to work, you would be able to work things out ?
I don't know the internals of 'sdist' but I think if there were a way to extend 'sdist' to use 'release' as well as 'version' then that might work. I would have to test that to see.
Well, can you define how sdist should behave exactly ?
Based on that discussion I can make a prototype for you to try out, then we can maybe propose in that mailing list a change to sdist
Regards Tarek
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno greno@verizon.net wrote:
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno greno@verizon.net wrote:
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages.
If I understand your problem correclty, if sdist would simply concatenate the version string and the release string to use it as a "source version" when it starts to work, you would be able to work things out ?
I don't know the internals of 'sdist' but I think if there were a way to extend 'sdist' to use 'release' as well as 'version' then that might work. I would have to test that to see.
Well, can you define how sdist should behave exactly ?
Based on that discussion I can make a prototype for you to try out, then we can maybe propose in that mailing list a change to sdist
Thanks Tarek. I think if it would do the same thing as bdist_rpm that it would be ok. bdist_rpm looks like it does VERSION-RELEASE (hyphen separator). So then doing this for 'sdist' I guess would produce a tarball name of foo-VERSION-RELEASE.tar.gz and an extracted directory of foo-VERSION-RELEASE. What this would allow then is for the 'version' string to stay at '5.0.0' and then the 'release' string to contain any pre-release information such as '0_rc1' and then the final release would contain '1' which is lexically superior to the '0_rc1'. I'm not sure though what other targets in distutils also use 'version' so I don't know if this would affect anything else. Also, I'm hoping this can be implemented as some kind of extension so that it can be made to work for existing installations as well.
Regards, Gerry
Gerry Reno wrote:
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 3:51 PM, Gerry Reno greno@verizon.net wrote:
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 2:12 AM, Gerry Reno greno@verizon.net wrote:
The problem is that there appears to be no way to get a coordinated behavior between 'sdist' and 'bdist_rpm' as far as version and release strings are concerned that will satisfy both tarballs and bdist packages.
If I understand your problem correclty, if sdist would simply concatenate the version string and the release string to use it as a "source version" when it starts to work, you would be able to work things out ?
I don't know the internals of 'sdist' but I think if there were a way to extend 'sdist' to use 'release' as well as 'version' then that might work. I would have to test that to see.
Well, can you define how sdist should behave exactly ?
Based on that discussion I can make a prototype for you to try out, then we can maybe propose in that mailing list a change to sdist
Thanks Tarek. I think if it would do the same thing as bdist_rpm that it would be ok. bdist_rpm looks like it does VERSION-RELEASE (hyphen separator). So then doing this for 'sdist' I guess would produce a tarball name of foo-VERSION-RELEASE.tar.gz and an extracted directory of foo-VERSION-RELEASE. What this would allow then is for the 'version' string to stay at '5.0.0' and then the 'release' string to contain any pre-release information such as '0_rc1' and then the final release would contain '1' which is lexically superior to the '0_rc1'. I'm not sure though what other targets in distutils also use 'version' so I don't know if this would affect anything else.
Updating my comment: Yes, and all the 'bdist' targets would have to do the same type of thing as 'bdist_rpm'. That is use the combination of VERSION-RELEASE.
Also, I'm hoping this can be implemented as some kind of extension so that it can be made to work for existing installations as well.
Regards, Gerry
Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
On Fri, Jan 9, 2009 at 4:42 PM, Gerry Reno greno@verizon.net wrote:
Thanks Tarek. I think if it would do the same thing as bdist_rpm that it would be ok. bdist_rpm looks like it does VERSION-RELEASE (hyphen separator). So then doing this for 'sdist' I guess would produce a tarball name of foo-VERSION-RELEASE.tar.gz and an extracted directory of foo-VERSION-RELEASE. What this would allow then is for the 'version' string to stay at '5.0.0' and then the 'release' string to contain any pre-release information such as '0_rc1' and then the final release would contain '1' which is lexically superior to the '0_rc1'. I'm not sure though what other targets in distutils also use 'version' so I don't know if this would affect anything else.
Updating my comment: Yes, and all the 'bdist' targets would have to do the same type of thing as 'bdist_rpm'. That is use the combination of VERSION-RELEASE.
Also, I'm hoping this can be implemented as some kind of extension so that it can be made to work for existing installations as well.
In other words, introduce it globally. That is a big change,
I think this could stay compatible with the previous installations as long as :
- if the release string is not specified, then it is not used at all, (unlike version which becomes 0.0 when not specified)
- the "Package-Version-Release" string is OK with the tools out there (I have to double-check on how setuptools and zc.buildout works on fragments to extract version numbers for instance)
I can't think of other issues at the moment, but I am pretty sure there are more
Regards Tarek
Tarek Ziadé wrote:
On Fri, Jan 9, 2009 at 4:42 PM, Gerry Reno greno@verizon.net wrote:
Thanks Tarek. I think if it would do the same thing as bdist_rpm that it would be ok. bdist_rpm looks like it does VERSION-RELEASE (hyphen separator). So then doing this for 'sdist' I guess would produce a tarball name of foo-VERSION-RELEASE.tar.gz and an extracted directory of foo-VERSION-RELEASE. What this would allow then is for the 'version' string to stay at '5.0.0' and then the 'release' string to contain any pre-release information such as '0_rc1' and then the final release would contain '1' which is lexically superior to the '0_rc1'. I'm not sure though what other targets in distutils also use 'version' so I don't know if this would affect anything else.
Updating my comment: Yes, and all the 'bdist' targets would have to do the same type of thing as 'bdist_rpm'. That is use the combination of VERSION-RELEASE.
Also, I'm hoping this can be implemented as some kind of extension so that it can be made to work for existing installations as well.
In other words, introduce it globally. That is a big change,
Yes, but it solves a big problem. I've seen this issue brought up on the fedora-devel mailing list several times. And Toshio has commented about it. The RPM-based distros appear to ignore 'bdist_rpm' and the distro packagers go about creating their own way of building python RPMS. And when I inquired about this, the pre-release versioning issue was the first thing that was mentioned. I think this was the showstopper that caused them to abandon 'bdist_rpm' because they couldn't find sanity between 'sdist' and 'bdist' command treatment of 'version' and 'release' strings. If this gets fixed now, I think they could be persuaded to consider using 'bdist_rpm' for building python packages which would bring a lot of consistency to the process between distros. (yes, I know distros all want to store things in different areas, but that's just a config/preference thing).
I think this could stay compatible with the previous installations as long as :
if the release string is not specified, then it is not used at all, (unlike version which becomes 0.0 when not specified)
the "Package-Version-Release" string is OK with the tools out there (I have to double-check on how setuptools and zc.buildout works on
fragments to extract version numbers for instance)
That sounds like it will work.
I'm looking forward to any prototype you could generate for this and I'll be glad to help test it.
Regards, Gerry
participants (2)
-
Gerry Reno -
Tarek Ziadé