Okay, here are the directions for bdist_rpm To run: ./setup.py bdist --format=rpm or ./setep.py bdist_rpm Will build an RPM using DistributionMetaData to get package information. Additional information can be put in the file 'package_data'. 'package_data' contains python code setting the following variables (variables marked with * are by default set from DistributionMetaData and do not be need assigned in package_data unless you need to override the values in .setup.py): *name - name of package *version - package version *summary - short summary of package *description - long summary of package summaries - a dictionary, keys are locales and values are summaries for that locale. descriptions - a dictionary, keys are locales and values are descriptions for that locale. *copyright - the license used by rpm release - the release of the rpm (for example '1mdk') group - the group the package belongs to vendor - the package's vendor packager - the package's packager *url - the package's URL doc - a list of string containing names of files and directories to put in the system's documentation directory. changelog - a string containing a properly formated changelog (I haven't tried it, but I'm pretty sure RPM will exit with an error if it is fed an improperly format changelog) Unless otherwise noted these must all be strings. There is no type checking yet, I would like people's opinion: should they be cast to strings or should an exception be raised if they are not strings? Or maybe some combination; for example releases are (especially on RedHat) often numerals, so it might be natural to allow a numeric value and cast it to a string, but a value like '8' would probably never make sense for something like 'copyright', so an exception would be raised. 'package_data's local namespace includes the variable 'package_type' which is set to 'rpm', this variable will hopefully allow 'package_data' to be used for all bdist_* commands . bdist_rpm works by making a spec file and saving it into the 'redhat/' directory (making the directory if necessary), it then produces a gzipped source tarball and runs `rpm -ta --clean` on the tarball. bdist_rpm leaves the spec file and tarball around since they would probably be wanted anyway. Here are the caveats I mentioned. If you have not configured rpm to allow you to build RPMs without being root you will have a directory and three files (the redhat/ directory and spec file, the MANIFEST file, and the tarball) saying around owned by root. The other caution is that the spec file must be included in MANIFEST.in, otherwise it will not be included in the tarball and therefore the tarball will not be buildable with rpm. The first will be solved with an option I am adding tonight which will just generate the spec file and tarball, the user can then su and run 'rpm -ta' on the tarball. The MANIFEST problem I don't know what is the best way to solve, should the spec file be automatically added like setup.py, should bdist_rpm add it to the MANIFEST itself, or should it remain the user's responsibility? If it is automatically included should package_data also be automatically included?, if a user alters the package in some way they will want to change the 'release' variable and regenerate the spec file, this will be easier if they have 'package_data' What is everybody's input on the above? -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python: http://pyncurses.sourceforge.ne
On 27 April 2000, Harry Henry Gebel said:
Will build an RPM using DistributionMetaData to get package information. Additional information can be put in the file 'package_data'. 'package_data' contains python code setting the following variables (variables marked with * are by default set from DistributionMetaData and do not be need assigned in package_data unless you need to override the values in .setup.py):
Yuck. I like the idea that all meta-data is supplied in the setup script, but some of this stuff is obviously RPM-specific so really shouldn't clutter up the main DistributionMetadata class/object. I think the answer is to put it in the mythical setup.cfg -- one more thing prodding me to go off and finish thinking through Distutils config files, argg! I'll let the 'package_data' file slide for now since I haven't supplied a viable alternative, but its days are numbered...
*name - name of package *version - package version *summary - short summary of package *description - long summary of package
Another yuck: description in setup.py -> summary in the .spec file, and long_description in setup.py -> description in the .spec file. Oh well: name conflicts happen, that's just life. I think the confusion can be localized to bdist_rpm.py, so it's not too bad.
doc - a list of string containing names of files and directories to put in the system's documentation directory.
This should be initialized to ["README"] or ["README.txt"] (whichever one happens to be supplied). The "sdist" command ought to make that information available, but there isn't really a clean way to get it now. Hmmm again.
Unless otherwise noted these must all be strings. There is no type checking yet, I would like people's opinion: should they be cast to strings or should an exception be raised if they are not strings?
I would say definitely do type-checking, and do it in the logical way: require that everything be a string except: * 'summaries' and 'descriptions' must be {string:string} dictionaries * 'release' can be a number or a string
'package_data's local namespace includes the variable 'package_type' which is set to 'rpm', this variable will hopefully allow 'package_data' to be used for all bdist_* commands .
I think the right way to do this is to put these "not-quite-fundamental" meta-data bits in the config file, in a "bdist" section (rather than a "bdist_rpm" section). But that points back to me getting back to the config file stuff....
bdist_rpm works by making a spec file and saving it into the 'redhat/' directory (making the directory if necessary), it then produces a gzipped source tarball and runs `rpm -ta --clean` on the tarball. bdist_rpm leaves the spec file and tarball around since they would probably be wanted anyway.
Why does the .spec file need a directory of its own? The .spec is tiny and essential; I think it belongs right in the distribution root, like a setup script or a makefile. Any temporary by-products of "bdist_rpm" belong somewhere in the build tree: build/rpm, perhaps? I haven't looked at the code yet, but I certainly hope that "bdist_rpm" uses "sdist" to create the source tarball -- if not, that should be fixed.
Here are the caveats I mentioned. If you have not configured rpm to allow you to build RPMs without being root you will have a directory and three files (the redhat/ directory and spec file, the MANIFEST file, and the tarball) saying around owned by root. The other caution is that the spec file must be included in MANIFEST.in, otherwise it will not be included in the tarball and therefore the tarball will not be buildable with rpm.
If you're talking about the standard source distribution tarball, I don't think it *has* to include the .spec file: after all, the point of the "bdist_rpm" command is to 1) generate a spec file, and 2) optionally run some "rpm -b" commands on it. Anyone who downloads the source distribution should be able to reconstruct the .spec file themselves using the "bdist_rpm" command. Of course, then the module developer has to be sure to include "package_data" in MANIFEST.in, but that file should go away to be replaced by setup.cfg, which *will* be one of the default files like README.txt and setup.py.
The first will be solved with an option I am adding tonight which will just generate the spec file and tarball, the user can then su and run 'rpm -ta' on the tarball.
How about another option to generate a source RPM? Not essential, but it just seems complete. Plus it's generally considered nice to provide source RPMs when you provide binary RPMs. I think it should be accessible like this: setup.py bdist --format=srpm or setup.py bdist_rpm --srpm Maybe --source-rpm or --source-only on that last one; not sure. Also, should the normal output of "bdist_rpm" be just a binary RPM, or both binary and source RPMs? If both by default, "--source-only" makes sense.
The MANIFEST problem I don't know what is the best way to solve, should the spec file be automatically added like setup.py, should bdist_rpm add it to the MANIFEST itself, or should it remain the user's responsibility? If it is automatically included should package_data also be automatically included?, if a user alters the package in some way they will want to change the 'release' variable and regenerate the spec file, this will be easier if they have 'package_data'
The default files that are included in MANIFEST are entirely up to the "sdist" command, which knows nothing about RPMs or other platform-specific distribution mechanisms. I maintain, though, that it's not necessary to include the .spec file in source distributions, so the issue is moot. I could be wrong... Off to actually read the code... Greg -- Greg Ward - Unix bigot gward@python.net http://starship.python.net/~gward/ One man's theology is another man's belly laugh.
On Tue, May 02, 2000 at 08:57:27PM -0400, Greg Ward wrote:
On 27 April 2000, Harry Henry Gebel said:
Will build an RPM using DistributionMetaData to get package information. Additional information can be put in the file 'package_data'. 'package_data' contains python code setting the following variables (variables marked with * are by default set from DistributionMetaData and do not be need assigned in package_data unless you need to override the values in .setup.py):
Yuck. I like the idea that all meta-data is supplied in the setup script, but some of this stuff is obviously RPM-specific so really shouldn't clutter up the main DistributionMetadata class/object. I think the answer is to put it in the mythical setup.cfg -- one more thing prodding me to go off and finish thinking through Distutils config files, argg!
I put it into a separate file to keep from putting RPM specific data in setup.py, I just picked package_data as the name because I did not know for sure what the current status of external configuration files in Distutils.
*name - name of package *version - package version *summary - short summary of package *description - long summary of package
Another yuck: description in setup.py -> summary in the .spec file, and long_description in setup.py -> description in the .spec file. Oh well: name conflicts happen, that's just life. I think the confusion can be localized to bdist_rpm.py, so it's not too bad.
Yea, I know. Unfortunately I didn't pick the names of the sections in a spec file. I wasn't sure whether to pick names that matched the Distutils meta-data or to pick names that matched spec file sections. I also thought about not allowing the packager to override the values that are already set in setup.py, if I disallowed this the whole issue would be moot. I am leaning more towards taking those variables out since, for example, overriding name or version would screw up the whole process (or at least the automation of the process). In the next patch all the variables that don't supply new data will probably be gone.
doc - a list of string containing names of files and directories to put in the system's documentation directory.
This should be initialized to ["README"] or ["README.txt"] (whichever one happens to be supplied). The "sdist" command ought to make that information available, but there isn't really a clean way to get it now. Hmmm again.
Will do.
Unless otherwise noted these must all be strings. There is no type checking yet, I would like people's opinion: should they be cast to strings or should an exception be raised if they are not strings?
I would say definitely do type-checking, and do it in the logical way: require that everything be a string except: * 'summaries' and 'descriptions' must be {string:string} dictionaries * 'release' can be a number or a string
Okay, I only left it out pending a decision on what form it should take, this will be in the next patch also.
'package_data's local namespace includes the variable 'package_type' which is set to 'rpm', this variable will hopefully allow 'package_data' to be used for all bdist_* commands .
I think the right way to do this is to put these "not-quite-fundamental" meta-data bits in the config file, in a "bdist" section (rather than a "bdist_rpm" section). But that points back to me getting back to the config file stuff....
bdist_rpm works by making a spec file and saving it into the 'redhat/' directory (making the directory if necessary), it then produces a gzipped source tarball and runs `rpm -ta --clean` on the tarball. bdist_rpm leaves the spec file and tarball around since they would probably be wanted anyway.
Why does the .spec file need a directory of its own? The .spec is tiny and essential; I think it belongs right in the distribution root, like a setup script or a makefile. Any temporary by-products of "bdist_rpm"
I am not familiar with any packaging systems other than rpm and debian (and with regards to debian I am only familiar with it to the extent that is possible from browsing through the PyNcurses 'debian/' directory (which was contributed by someone else)). Not being familiar with other packaging systems I was a little afraid that another packaging system might have the same spec file naming convention as rpm and I didn't want them setting on each other. If this is not a concern I can change it to use the distribution root instead. I also thought it would fit in with the 'debian/' directory that the debian packaging system seems to use.
tarball) saying around owned by root. The other caution is that the spec file must be included in MANIFEST.in, otherwise it will not be included in the tarball and therefore the tarball will not be buildable with rpm.
If you're talking about the standard source distribution tarball, I don't think it *has* to include the .spec file: after all, the point of the "bdist_rpm" command is to 1) generate a spec file, and 2) optionally run some "rpm -b" commands on it. Anyone who downloads the source distribution should be able to reconstruct the .spec file themselves using the "bdist_rpm" command. Of course, then the module developer has
The problem with not including the spec file is that I am using `rpm -ta` command rather than `rpm -ba` command. I am using this to get around the fact that the different distributions put rpm related files by default in different directories from each other, plus many users don't like to build RPMs as root and so set up their own directories for use with rpm. To use `rpm -ba` I would have to determine the location of the 'SOURCES/' directory and the only way I could figure out to do this was parsing the rpmrc, and this I did not want to do. Maybe I am missing an rpm command that spits this information out or allows you to define a specific source directory on the command line. `rpm -ta` takes a tarball containing a spec file and builds the RPM from that tarball and spec file. When the source RPM is produced the original tarball is placed in as the source file. Therefore, any source RPM produced with `rpm -ta` will contain a spec file in it's source tarball. The convention in rpm (and this is repeatedly stressed in rpm development materials) is to use pristine sources ie. the exact source distribution you would get if you went and downloaded it from the author's site. If the spec file is not included in the standard source distribution then any source RPM produced with `rpm -ta` will by definition be in violation of this convention.
How about another option to generate a source RPM? Not essential, but it just seems complete. Plus it's generally considered nice to provide source RPMs when you provide binary RPMs. I think it should be accessible like this:
setup.py bdist --format=srpm or setup.py bdist_rpm --srpm
Maybe --source-rpm or --source-only on that last one; not sure.
Also, should the normal output of "bdist_rpm" be just a binary RPM, or both binary and source RPMs? If both by default, "--source-only" makes sense.
At the moment the default is to build both binary and source. rpm goes through the whole process whether you are building a binary or source rpm or both so I figured it would be better to just do both. Of course if there is a desire out there to only build one or the other I could easily add '--source-only' and '--binary-only' options that just call `rpm -ts` and `rpm -tb` instead of `rpm -ta`. Did you read my post regarding CFLAGS, and what do you feel is the best way to go on that issue? -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python: http://pyncurses.sourceforge.ne
On Wed, 3 May 2000, Harry Henry Gebel wrote:
Subject: Re: [Distutils] Instructions for bdist_rpm
[snipping and probably losing who said what....]
bdist_rpm works by making a spec file and saving it into the 'redhat/' directory (making the directory if necessary), it then produces a gzipped source tarball and runs `rpm -ta --clean` on the tarball. bdist_rpm leaves the spec file and tarball around since they would probably be wanted anyway.
A couple of concerns: Redhat uses the /usr/src/redhat convention, Mandrake uses /usr/src/RPM so it's even worse than you noted. You could use rpm --showrc, but you'd probably end up (re)writing a parser to handle RPM macros, which would be just as bad as parsing the rpmrc (once you fond it). The -ta option is interesting. How does it handle patches? (see later on...)
Why does the .spec file need a directory of its own? The .spec is tiny and essential; I think it belongs right in the distribution root, like a setup script or a makefile. Any temporary by-products of "bdist_rpm"
I am not familiar with any packaging systems other than rpm and debian (and with regards to debian I am only familiar with it to the extent that is possible from browsing through the PyNcurses 'debian/' directory (which was contributed by someone else)). Not being familiar with other packaging systems I was a little afraid that another packaging system might have the same spec file naming convention as rpm and I didn't want them setting on each other. If this is not a concern I can change it to use the distribution root instead. I also thought it would fit in with the 'debian/' directory that the debian packaging system seems to use.
I don't know debian well, but pkgtool (Solaris and others) includes the spec file type info in the files pkginfo and pkgproto, with other script files also allowed. They are annoyingly ALWAYS THOSE NAMES so they CANNOT go into the same directory. By convention, they are not usually even included in the packages. HP-UX uses .psf files, but I think that's a convention, not a requirement.
The problem with not including the spec file is that I am using `rpm -ta` command rather than `rpm -ba` command. I am using this to get around the fact that the different distributions put rpm related files by default in different directories from each other, plus many users don't like to build RPMs as root and so set up their own directories for use with rpm. To use `rpm -ba` I would have to determine the location of the 'SOURCES/' directory and the only way I could figure out to do this was parsing the rpmrc, and this I did not want to do. Maybe I am missing an rpm command that spits this information out or allows you to define a specific source directory on the command line.
But still, don't you build the spec file from the distutil information anyway? That's going to be true of whatever package format the user requires, which, if it's not RPM the spec file is extraneous. I agree that figuring out the where's and what's of which rpm is going to be a nightmare...BUT, you can create a spec file with internal definitions for RPM_BUILD_ROOT, RPM_SOURCE_DIR, RPM_BUILD_DIR, etc, that does all the dirty work in /tmp, /var/tmp, or wherever, leaving only the final binary and/or source RPMS in the user's home directory. (That's my theory and I'm sticking to it untill someone points out an obvious error ;-). Anywhat, I strongly agree with supporting the creation of RPM's by non-root users. I just consider it safer, since it's pretty much impossible to step on a functioning installation.
`rpm -ta` takes a tarball containing a spec file and builds the RPM from that tarball and spec file. When the source RPM is produced the original tarball is placed in as the source file. Therefore, any source RPM produced with `rpm -ta` will contain a spec file in it's source tarball.
The convention in rpm (and this is repeatedly stressed in rpm development materials) is to use pristine sources ie. the exact source distribution you would get if you went and downloaded it from the author's site. If the spec file is not included in the standard source distribution then any source RPM produced with `rpm -ta` will by definition be in violation of this convention.
BUT, they also support inclusion and application of patches with the "pristine" sources in order to support the inevitable modifications of dealing with multiple distributions and platforms. (By the way, for those that don't know, RPM does run on non-Linux UNIX'es. It's not pretty trying to get dependency interaction with stuff already installed, 'though...) So the big question is does the -ta option support the inclusion of patch sets?
At the moment the default is to build both binary and source. rpm goes through the whole process whether you are building a binary or source rpm or both so I figured it would be better to just do both. Of course if there is a desire out there to only build one or the other I could easily add '--source-only' and '--binary-only' options that just call `rpm -ts` and `rpm -tb` instead of `rpm -ta`.
I positively vote either-or-whatever! The only consideration I see is if someone is extremely space tight. It takes moments longer to produce the source rpm. mwa
On Wed, May 03, 2000 at 05:59:32PM -0400, Mark W. Alexander wrote:
A couple of concerns: Redhat uses the /usr/src/redhat convention, Mandrake uses /usr/src/RPM so it's even worse than you noted. You could use rpm --showrc, but you'd probably end up (re)writing a parser to handle RPM macros, which would be just as bad as parsing the rpmrc (once you fond it). The -ta option is interesting.
This was exactly my point.
But still, don't you build the spec file from the distutil information anyway? That's going to be true of whatever package format the user requires, which, if it's not RPM the spec file is extraneous. I agree that figuring out the where's and what's of which rpm is going to be a nightmare...BUT, you can create a spec file with internal definitions for RPM_BUILD_ROOT, RPM_SOURCE_DIR, RPM_BUILD_DIR, etc, that does all the dirty work in /tmp, /var/tmp, or wherever, leaving only the final binary and/or source RPMS in the user's home directory. (That's my theory and I'm sticking to it untill someone points out an obvious error ;-).
I did not specify them in the spec file because I do not want them being hard-coded as going places that the user does not expect. What I am going to try is to dump the rc with `rpm --showrc` and then tag new definitions to the end of the a temporory file and use that file as an rc file, hopefully that will work. Believe me, I would rather use `rpm -ba` rather than `rpm -ta` for a lot of reasons, if I could figure out a way to make it work without stepping on source RPM user's expectations.
BUT, they also support inclusion and application of patches with the "pristine" sources in order to support the inevitable modifications of dealing with multiple distributions and platforms. (By the way, for those that don't know, RPM does run on non-Linux UNIX'es. It's not pretty trying to get dependency interaction with stuff already installed, 'though...) So the big question is does the -ta option support the inclusion of patch sets?
`rpm -ta` does not support patches, but I do not think that is a problem because I don't plan to put patch support into bdist_rpm. People who are producing patched RPMs will have to add the patches to the spec file by hand, this is a simple process that involves adding two lines to the spec file per patch and is one of the reasons I tried to keep the spec file as clean as possible. -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python: http://pyncurses.sourceforge.net
On Thu, 4 May 2000, Harry Henry Gebel wrote:
`rpm -ta` does not support patches, but I do not think that is a problem because I don't plan to put patch support into bdist_rpm. People who are producing patched RPMs will have to add the patches to the spec file by hand, this is a simple process that involves adding two lines to the spec file per patch and is one of the reasons I tried to keep the spec file as clean as possible.
Sold. Upon reflection, given the context of Distutils, patch support does seem kinda silly. That pretty much eradicates the other issues of using -t? vs. -b?. I'll step out of your way now... mwa
On Thu, May 04, 2000 at 09:44:38AM -0400, Mark W. Alexander wrote:
On Thu, 4 May 2000, Harry Henry Gebel wrote:
`rpm -ta` does not support patches, but I do not think that is a problem because I don't plan to put patch support into bdist_rpm. People who are producing patched RPMs will have to add the patches to the spec file by hand, this is a simple process that involves adding two lines to the spec file per patch and is one of the reasons I tried to keep the spec file as clean as possible. Sold. Upon reflection, given the context of Distutils, patch support does seem kinda silly. That pretty much eradicates the other issues of using -t? vs. -b?. I'll step out of your way now...
My own personal reason is so I can add an option to use bztar for the source code; rpm supports bzip2 with `rpm -ba` but not `rpm -ta`. And I think Greg is right that they shouldn't have to include the spec file in the RPM, it's just a requirement as long as I am using `rpm -ta`, switching to `rpm -ba` would eliminate that to. -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python: http://pyncurses.sourceforge.ne
On 04 May 2000, Harry Henry Gebel said:
My own personal reason is so I can add an option to use bztar for the source code; rpm supports bzip2 with `rpm -ba` but not `rpm -ta`. And I think Greg is right that they shouldn't have to include the spec file in the RPM, it's just a requirement as long as I am using `rpm -ta`, switching to `rpm -ba` would eliminate that to.
Sounds like you're moving in the right direction to me, so I assume another version of the patch is forthcoming. For the record, I think all are agreed here: no need to support patches in Distutils-generated RPMs, better to use -b than -t, and preferable not to require including the .spec file in the source distribution. I don't see a burning need for bzip2 support, but if you can work it in cleanly (like with the "sdist" command) go ahead. Before you do send another version of the patch, though, I have one small request: lowercase_method_names please, not camelCase! It's the convention I've used throughout the Distutils (and in pretty much every line of C, Python, and Perl I've written in the last 4 or 5 years), and I'd rather keep the code consistent. Thanks! Greg -- Greg Ward - nerd gward@python.net http://starship.python.net/~gward/ Sure, I'm paranoid... but am I paranoid ENOUGH?
On Thu, May 04, 2000 at 10:22:22PM -0400, Greg Ward wrote:
My own personal reason is so I can add an option to use bztar for the source code; rpm supports bzip2 with `rpm -ba` but not `rpm -ta`. And I
On 04 May 2000, Harry Henry Gebel said: the .spec file in the source distribution. I don't see a burning need for bzip2 support, but if you can work it in cleanly (like with the "sdist" command) go ahead.
Yes I am planning on using sdist to generate the bzipped tarball, and it is going to be an option, leaving gzip as the default. The main reason is to shrink the size of the source RPM (it will have no effect on the binary RPM since rpm has no options (yet) supporting changing compression methods in binary RPMs).
Before you do send another version of the patch, though, I have one small request: lowercase_method_names please, not camelCase! It's the convention I've used throughout the Distutils (and in pretty much every line of C, Python, and Perl I've written in the last 4 or 5 years), and I'd rather keep the code consistent. Thanks!
Sorry, force of habit. I noticed the convention used in Distutils and made a note to use it, but forgot the note! The names will be changed in the next patch. -- Harry Henry Gebel, Senior Developer, Landon House SBS West Dover Hundred, Delaware PyNcurses ncurses binding for Python: http://pyncurses.sourceforge.ne
participants (3)
-
Greg Ward -
Harry Henry Gebel -
Mark W. Alexander