Re: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions
Tarek -
Thanks.
I got it working and want to document some of my findings. I'm cc:'ing the Fedora Python list so they can take these issues into account in their Python packaging instructions. My findings are as follows:
1. The source code management system was git. I needed to install setuptools-git to get files recognized that were being maintained under git.
2. I also needed to establish a MANIFEST.in file to ensure all relevant files were included.
3. The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't work if there are directories involved. This is a documented "gotcha" in http://fedoraproject.org/wiki/How_to_create_an_RPM_package
They also advise avoiding use of INSTALLED_FILES.
Here is what I used in the spec file (it had to be edited for the directories to be included in docs):
%install python setup.py install --root=$RPM_BUILD_ROOT cd $RPM_BUILD_ROOT mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples cd %{_builddir}/%{name}-%{version} cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs cp -rp examples/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples
%clean rm -rf $RPM_BUILD_ROOT
%files %defattr(-,root,root) %{_defaultdocdir}/%{name}-%{version}/ %{python_sitelib}/%{name}/ %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/
4. The overall approach I used was to run
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-only
That brought in items such as the description specified in setup.py. I then moved the spec file up a directory level from dist, edited in the above parts relevant to the doc files, and finally ran
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2
--spec-file=SPECFILE
I also found it very useful to actually run
python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file SPECFILE >buildout 2>&1
Recording the build output enabled me to find details of errors that occurred during the build. There were a number of those, and the output flies by too quickly to notice them for diagnosis of problems that need to be fixed. (an approach I've also used in the past is to tee the output, so I can see if the build was successful when it completes). Also, the -k option came in useful in figuring out the documentation-related fixes to the spec file, because it helped seeing what was included in the build and what was available in the build but not getting copied to the install.
5. I inquired about how the project produced its sdist files for pypi, and was told that they had so many problems related to their switch to git that they simply did a gzipped tar of their repository.
6. The project used a different extension on their README file from the expected values of README and README.txt. I needed to change that to README.txt to get it to process properly.
Again, thanks.
Stan Klein
On Fri, November 11, 2011 6:00 am, Tarek Ziad? ziade.tarek@gmail.com wrote:
Message: 3 Date: Thu, 10 Nov 2011 18:42:08 +0100 From: Tarek Ziad? ziade.tarek@gmail.com To: "Stanley A. Klein" sklein@cpcug.org Cc: distutils-sig@python.org Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions
On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein sklein@cpcug.org wrote:
Tarek -
I downloaded pypi2rpm and built it using bdist_rpm.
you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need to build, just install
?The instructions on how to use it are rather thin.
Yes there's no doc, as its mostly use in our own build tools for now.
My best guess is to run it in spec-only mode, modify the spec, and then run it with the modified spec specified. I tried to get a help listing but that was also think and I couldn't quite figure out how to run it.
Could you provide further information on how to use it.
Thanks.
Once it's installed you can build a rpm in your project, using:
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file=SPECFILE
(there are other options you can find with --help)
Stan Klein
On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote:
On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat pnasrat@gmail.com wrote:
I don't think bdist_rpm should track vendor packaging requirements, purely as those recommendations may change faster than the release process of distutils. I also believe bdist_rpm may be going away in the future:
Yes I confirm this. We removed it in packaging because we believe it should be maintained by the RPM communities -- with their own release cycles etc.
FWIW I have a custom version in the pypi2rpm project where I just feed a .spec file to the bdist_rpm command, so I can do proper RHEL or Fedora packaging.
For Fedora have you considered rpmdev-newspec which can creates a templated python spec file for your packages.
http://fedoraproject.org/wiki/How_to_create_an_RPM_package
Paul
On 8 November 2011 21:40, Stanley A. Klein sklein@cpcug.org wrote:
I will need to build some Python packages for Fedora and Centos. ?The spec file produced by bdist_rpm automatically includes the statement %files -f INSTALLED_FILES
The Fedora Python packaging instruction includes a recommendation to avoid use of INSTALLED_FILES and provides some alternatives. ?That is the first incompatibility I've encountered, but there may be more.
The bdist_rpm code probably should be changed to enable compatibility. Meanwhile, is there a workaround?
Stan Klein
Thanks for the feedback Stanley !
If you agree, I'll add your document into the pypi2rpm/bdist_rpm2 project, for other people
Cheers Tarek
On Sat, Nov 26, 2011 at 8:00 PM, Stanley A. Klein sklein@cpcug.org wrote:
Tarek -
Thanks.
I got it working and want to document some of my findings. I'm cc:'ing the Fedora Python list so they can take these issues into account in their Python packaging instructions. My findings are as follows:
- The source code management system was git. I needed to install
setuptools-git to get files recognized that were being maintained under git.
- I also needed to establish a MANIFEST.in file to ensure all relevant
files were included.
- The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't
work if there are directories involved. This is a documented "gotcha" in http://fedoraproject.org/wiki/How_to_create_an_RPM_package
They also advise avoiding use of INSTALLED_FILES.
Here is what I used in the spec file (it had to be edited for the directories to be included in docs):
%install python setup.py install --root=$RPM_BUILD_ROOT cd $RPM_BUILD_ROOT mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples cd %{_builddir}/%{name}-%{version} cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs cp -rp examples/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples
%clean rm -rf $RPM_BUILD_ROOT
%files %defattr(-,root,root) %{_defaultdocdir}/%{name}-%{version}/ %{python_sitelib}/%{name}/ %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/
- The overall approach I used was to run
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-only
That brought in items such as the description specified in setup.py. I then moved the spec file up a directory level from dist, edited in the above parts relevant to the doc files, and finally ran
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2
--spec-file=SPECFILE
I also found it very useful to actually run
python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file SPECFILE >buildout 2>&1
Recording the build output enabled me to find details of errors that occurred during the build. There were a number of those, and the output flies by too quickly to notice them for diagnosis of problems that need to be fixed. (an approach I've also used in the past is to tee the output, so I can see if the build was successful when it completes). Also, the -k option came in useful in figuring out the documentation-related fixes to the spec file, because it helped seeing what was included in the build and what was available in the build but not getting copied to the install.
- I inquired about how the project produced its sdist files for pypi,
and was told that they had so many problems related to their switch to git that they simply did a gzipped tar of their repository.
- The project used a different extension on their README file from the
expected values of README and README.txt. I needed to change that to README.txt to get it to process properly.
Again, thanks.
Stan Klein
On Fri, November 11, 2011 6:00 am, Tarek Ziad? ziade.tarek@gmail.com wrote:
Message: 3 Date: Thu, 10 Nov 2011 18:42:08 +0100 From: Tarek Ziad? ziade.tarek@gmail.com To: "Stanley A. Klein" sklein@cpcug.org Cc: distutils-sig@python.org Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions
On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein sklein@cpcug.org wrote:
Tarek -
I downloaded pypi2rpm and built it using bdist_rpm.
you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need to build, just install
?The instructions on how to use it are rather thin.
Yes there's no doc, as its mostly use in our own build tools for now.
My best guess is to run it in spec-only mode, modify the spec, and then run it with the modified spec specified. I tried to get a help listing but that was also think and I couldn't quite figure out how to run it.
Could you provide further information on how to use it.
Thanks.
Once it's installed you can build a rpm in your project, using:
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file=SPECFILE
(there are other options you can find with --help)
Stan Klein
On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote:
On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat pnasrat@gmail.com wrote:
I don't think bdist_rpm should track vendor packaging requirements, purely as those recommendations may change faster than the release process of distutils. I also believe bdist_rpm may be going away in the future:
Yes I confirm this. We removed it in packaging because we believe it should be maintained by the RPM communities -- with their own release cycles etc.
FWIW I have a custom version in the pypi2rpm project where I just feed a .spec file to the bdist_rpm command, so I can do proper RHEL or Fedora packaging.
For Fedora have you considered rpmdev-newspec which can creates a templated python spec file for your packages.
http://fedoraproject.org/wiki/How_to_create_an_RPM_package
Paul
On 8 November 2011 21:40, Stanley A. Klein sklein@cpcug.org wrote:
I will need to build some Python packages for Fedora and Centos. ?The spec file produced by bdist_rpm automatically includes the statement %files -f INSTALLED_FILES
The Fedora Python packaging instruction includes a recommendation to avoid use of INSTALLED_FILES and provides some alternatives. ?That is the first incompatibility I've encountered, but there may be more.
The bdist_rpm code probably should be changed to enable compatibility. Meanwhile, is there a workaround?
Stan Klein
Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Tarek -
Of course. That's why I documented it. However, in a future version of bdist_rpm it would be good if some of these issues were accommodated rather than requiring manual editing and processing.
Stan Klein
On Sat, November 26, 2011 4:24 pm, Tarek Ziadé wrote:
Thanks for the feedback Stanley !
If you agree, I'll add your document into the pypi2rpm/bdist_rpm2 project, for other people
Cheers Tarek
On Sat, Nov 26, 2011 at 8:00 PM, Stanley A. Klein sklein@cpcug.org wrote:
Tarek -
Thanks.
I got it working and want to document some of my findings. I'm cc:'ing the Fedora Python list so they can take these issues into account in their Python packaging instructions. My findings are as follows:
- The source code management system was git. I needed to install
setuptools-git to get files recognized that were being maintained under git.
- I also needed to establish a MANIFEST.in file to ensure all relevant
files were included.
- The setup.cfg statement under [bdist_rpm] of "doc_files =" doesn't
work if there are directories involved. This is a documented "gotcha" in http://fedoraproject.org/wiki/How_to_create_an_RPM_package
They also advise avoiding use of INSTALLED_FILES.
Here is what I used in the spec file (it had to be edited for the directories to be included in docs):
%install python setup.py install --root=$RPM_BUILD_ROOT cd $RPM_BUILD_ROOT mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs mkdir -p %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples cd %{_builddir}/%{name}-%{version} cp -p *.txt %{buildroot}%{_defaultdocdir}/%{name}-%{version}/ cp -rp docs/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/docs cp -rp examples/ %{buildroot}%{_defaultdocdir}/%{name}-%{version}/examples
%clean rm -rf $RPM_BUILD_ROOT
%files %defattr(-,root,root) %{_defaultdocdir}/%{name}-%{version}/ %{python_sitelib}/%{name}/ %{python_sitelib}/%{name}-%{version}-py%{python_version}.egg-info/
- The overall approach I used was to run
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-only
That brought in items such as the description specified in setup.py. I then moved the spec file up a directory level from dist, edited in the above parts relevant to the doc files, and finally ran
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2
--spec-file=SPECFILE
I also found it very useful to actually run
python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file SPECFILE >buildout 2>&1
Recording the build output enabled me to find details of errors that occurred during the build. There were a number of those, and the output flies by too quickly to notice them for diagnosis of problems that need to be fixed. (an approach I've also used in the past is to tee the output, so I can see if the build was successful when it completes). Also, the -k option came in useful in figuring out the documentation-related fixes to the spec file, because it helped seeing what was included in the build and what was available in the build but not getting copied to the install.
- I inquired about how the project produced its sdist files for pypi,
and was told that they had so many problems related to their switch to git that they simply did a gzipped tar of their repository.
- The project used a different extension on their README file from the
expected values of README and README.txt. I needed to change that to README.txt to get it to process properly.
Again, thanks.
Stan Klein
On Fri, November 11, 2011 6:00 am, Tarek Ziad? ziade.tarek@gmail.com wrote:
Message: 3 Date: Thu, 10 Nov 2011 18:42:08 +0100 From: Tarek Ziad? ziade.tarek@gmail.com To: "Stanley A. Klein" sklein@cpcug.org Cc: distutils-sig@python.org Subject: Re: [Distutils] Compatibility of bdist_rpm with Fedora packaging instructions
On Thu, Nov 10, 2011 at 6:11 PM, Stanley A. Klein sklein@cpcug.org wrote:
Tarek -
I downloaded pypi2rpm and built it using bdist_rpm.
you built it ? pypi2rpm is a script and a bidist_rpm2 command, no need to build, just install
?The instructions on how to use it are rather thin.
Yes there's no doc, as its mostly use in our own build tools for now.
My best guess is to run it in spec-only mode, modify the spec, and then run it with the modified spec specified. I tried to get a help listing but that was also think and I couldn't quite figure out how to run it.
Could you provide further information on how to use it.
Thanks.
Once it's installed you can build a rpm in your project, using:
$ python setup.py --command-packages=pypi2rpm.command bdist_rpm2 --spec-file=SPECFILE
(there are other options you can find with --help)
Stan Klein
On Thu, November 10, 2011 5:06 am, Tarek Ziad? wrote:
On Thu, Nov 10, 2011 at 10:04 AM, Paul Nasrat pnasrat@gmail.com wrote:
I don't think bdist_rpm should track vendor packaging requirements, purely as those recommendations may change faster than the release process of distutils. I also believe bdist_rpm may be going away in the future:
Yes I confirm this. We removed it in packaging because we believe it should be maintained by the RPM communities -- with their own release cycles etc.
FWIW I have a custom version in the pypi2rpm project where I just feed a .spec file to the bdist_rpm command, so I can do proper RHEL or Fedora packaging.
For Fedora have you considered rpmdev-newspec which can creates a templated python spec file for your packages.
http://fedoraproject.org/wiki/How_to_create_an_RPM_package
Paul
On 8 November 2011 21:40, Stanley A. Klein sklein@cpcug.org wrote: > I will need to build some Python packages for Fedora and Centos. > ?The > spec > file produced by bdist_rpm automatically includes the statement > %files -f INSTALLED_FILES > > The Fedora Python packaging instruction includes a recommendation > to > avoid > use of INSTALLED_FILES and provides some alternatives. ?That is the > first > incompatibility I've encountered, but there may be more. > > The bdist_rpm code probably should be changed to enable > compatibility. > Meanwhile, is there a workaround? > > > Stan Klein
Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
-- Tarek Ziadé | http://ziade.org
participants (2)
-
Stanley A. Klein -
Tarek Ziadé