Hi folks, Problem: The latest Numeric release on the web site is 20.3. The latest with an RPM is 20.1, and that RPM has a problem: it creates a directory in the system root directory. Paul D. says he will implement a solution but doesn't have the experience with RPMs (or the time) to find the problem quickly. I haven't dealt with building Python packages or distutils (is distutils a separate thing or part of Python?) at all. Can someone with the relevant experience fix the current problem and help Paul implement the solution so he can post current RPMs that install right? Ditto anyone who knows how to make packages for Debian, Solaris, and other popular package managers. Rationale: As I've mentionned previously, I'm getting an increasing number of queries from astronomers who want to play with Numeric. At this stage many of the converts will be application code contributors who will help build a library of discipline-specific routines. In talking to these people, I am finding them less than patient with the good 'ol tarball (a position I take myself, following the experience of maintaining the Clue Files, see ftp://oobleck.astro.cornell.edu/pub/clues.tar.gz). To them, it's not serious software if it isn't prepared under their system's installation manager. We need these (very) early adopters, so I think that having a current Numeric RPM for i386 Linux (and the equivalent for i386 Debian GNU/Linux and Solaris Sparc architectures, if someone knows how to build them) would be a Good Thing. Trivial install -> more users, more users -> more volunteers and more contributed code. Also, it would be more consistent with the RPM naming scheme to call the RPM "python-Numeric" (or "python-numeric", or even "numpy") rather than just "Numeric". If that's hard or philosophically undesirable, don't bother, but the name has changed a few times, so I hope it isn't a big deal. Sysadmins have to deal with more than 1000 packages now, and knowing what a package is just by looking at the name is a big help. Also, you can do things like 'rpm -qa | grep python' and get a list of all the python-related packages on your system. "Numeric" is too general outside the context of Python. All of the above goes for Numarray, when its developers are ready for the community at large to start writing code that uses it. Thanks, --jh--
Hello All, Debian (unstable -- which is actually quite stable) provides slightly more up-to-date packages of numeric (20.2) and its extensions. You can find them here: http://packages.debian.org/unstable/math/python-numeric.html http://packages.debian.org/unstable/interpreters/python-numeric-ext.html For a quick hack at a more up-to-date RPM, it _might_ be possible to use alien to convert the *.debs to *.rpms... Scott PS: As an astronomer myself, I am also seeing an increasing interest in my collegues towards python and numeric (especially since I keep preaching the Python gospel to them ;) On January 2, 2002 02:29 pm, Joe Harrington wrote:
-- Scott M. Ransom Address: McGill Univ. Physics Dept. Phone: (514) 398-6492 3600 University St., Rm 338 email: ransom@physics.mcgill.ca Montreal, QC Canada H3A 2T8 GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
"SR" == Scott Ransom <ransom@physics.mcgill.ca> writes:
SR> Hello All, Debian (unstable -- which is actually quite stable) SR> provides slightly more up-to-date packages of numeric (20.2) SR> and its extensions. You can find them here: SR> http://packages.debian.org/unstable/math/python-numeric.html SR> http://packages.debian.org/unstable/interpreters/python-numeric-ext.html Nothing like 2 days to make it now 20.3 in Debian unstable/Sid. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ -------------- FHCRC: M-W: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: T-Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX Rosen: (Mullins' Lab) Fridays, and I'm unreachable except by email.
Hi, Try python setup.py bdist_rpm Works like a charm. The same for the subdirectories/subpackages (maybe after installation of the main RPMs). Gerard PS: it is impossible to provide binary RPMs, because there are too many different Linux distributions. On Wednesday 02 January 2002 20:29, Joe Harrington wrote:
Joe Harrington <jh@oobleck.astro.cornell.edu> writes:
If someone can give a precise description of the problem, I'll look at it, I think I have done enough RPMs to find my way... The RPMs generated by Distutils are often good enough, but sometimes need some manual cleanup.
distutils (is distutils a separate thing or part of Python?) at all.
It is a part of Python since Python 1.6.
installation manager. We need these (very) early adopters, so I think that having a current Numeric RPM for i386 Linux (and the equivalent
I agree in principle, but providing Linux binary RPMs is becoming more and more messy: it depends on the Python version and on the distribution, sometimes even the distribution version number. That's a lot of RPMs. Source RPMs are easier, with some care they should work everywhere, but installation requires a compiler plus some of the "-devel" packages that not everyone has.
Agreed. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
In the exchange below, my query is >>, Gerard's response is >, and my present response to that is unquoted.
Yes, mine go into /usr/lib/python2.1
Well, the paths are figured out in /usr/lib/python2.1/distutils/sysconfig.py, using sys.prefix or sys.exec_prefix.
Or that it resides on an NFS mounted volume??? Or some python startup file that messes sys.(exec_)prefix.
Ok, Paul, Konrad, is this what you need?
Every combination of Distro-Version-Python-Version would require its own binary RPM.
If you consider Microsoft Windows one OS, Red Hat Linux another, Debian GNU/Linux a third, and so on, the distinction goes away. The fact that Microsoft has a monopoly is definitely convenient to producers. However, as users, we lose and shouldn't support it.
(B) The other possibility is to include a generic python-numeric.spec file (you build an RPM with rpm -ba python-numeric.spec). I can adapt mine.
Gerard
PS: do you have a RPM based Linux, yourself?
Yes, I run Red Hat. So do the vast majority of US astronomers I have spoken to. I understand that SuSE is equally popular in Europe. A few other dists just copy one of them and add a few packages, and so don't need separate RPMs of their own. If we configure RPMs with the version of python included in these dists, we're not talking about a large number of packages, after all, to get 98% of the users or better. I think you are onto a solution: We build RPMs (and .deb files for Debian, if we think there are enough Debian users to make it worthwhile, and the same for Solaris if it uses a different package manager) for the current release of each popular distribution, with that release's python. That's maybe 4 binary packages per Numeric release (Red Hat, SuSE, Debian, Solaris). We also make a source RPM and distribute with it instructions for building a new binary RPM with appropriately-named version ID (e.g., python-numeric-20.4py2.2glibc6-1). We solicit users of unsupported combinations of OS and Python version to contribute those RPMs along with a simple ASCII form specifying what kind of system made it, contact info, etc. I predict we will not get a huge number of such submissions from outside this list, as the vast majority of Numeric users who upgrade their Python when a new Python release comes out (as opposed to a new OS release) are developers who don't care about the RPM anyway. The ultimate solution is to get this thing incorporated into the python core. --jh--
I can confirm that the Python I use is in /pcmdi/dubois/python. I *never* build into /usr or /usr/local. There are lots of people like me that have multiple pythons around with other stuff added in. I suppose what was bothering people was having to use --prefix because the default will not work because my Python is not in some standard place (said standard place seeming to me to be a mythical location given how I see people actually working). I continue to be less than impressed by RPMs. Every time I try one I get version conflicts with other software.
I *never* build into /usr or /usr/local.
Good! You should leave /usr for your package manager to deal with. However, you should *release* software to install there, which you can with RPM without having to build it in /usr. /usr/local is yours to play with. The only thing in mine is IDL. (...which is why I'd like to delete it!)
There was no documentation anywhere in the download area suggesting the use of --prefix. Even if there were, RPMs get collected and put in archives (rpmfind.net) without any other associated docs. They should always install into the system following the Linux file system standards. Nobody expects to have to configure an RPM. They're just supposed to work, and the vast majority just do.
There are lots of people like me that have multiple pythons around with other stuff added in.
There are lots if *developers* like you. The current group of Numpy users is demographically distinct from the future user base, if Numpy becomes a player for numerical work in science and engineering. We will have many users who have never written a line of C or C++ or FORTRAN and who will run screaming at the mention of the word "tar". We will have high school teachers and students. We will have lots of people who have zero time and zero tolerance for learning unrelated topics like system administration or software tweaking. If we don't care about them, they won't use our software.
I continue to be less than impressed by RPMs. Every time I try one I get version conflicts with other software.
I understand your frustration with the bad RPMs, but it's not the fault of RPM, it's the fault of either the package developer or the person installing the package. If the package developer links against a library that is itself under rapid development, there is a problem waiting to happen. It's worse if that library is included in some or all distributions. The simple solution is to statically link that library, or rename it and include it in the package. If the user has an out-of-date system, it's also trouble. If we build against the C library and python that are in the OS releases, we're in good shape. I don't think we need to bend over backwards to provide RPMs for people who install their own Python, unless a new version of Numeric requires use of an updated Python RPM. Very few (maybe 0.5%?) "mass-market" users will install their own Python. The experts who do can handle a tarball install, or one will build an RPM and distribute it if it's a particularly important release. The conflict problems pale in comparison to the problems with manually building all packages you want to install, particularly if you don't know how to fix makefile problems or don't have a compiler. If you're only installing a few packages, it's not that big a deal. If you're installing 90, it's a month's work by hand or a day with RPM, if you don't know the install procedure for the software beforehand. See ftp://oobleck.astro.cornell.edu/pub/clues.tar.gz for my attempt to tame this problem. As you can see from the dates of the files therein, I abandonned maintaining each package page as it came out under RPM. It was just impossible to keep up. It's trivial under RPM. Plus, I can uninstall! What matters is what the general users want, and they have spoken loudly in favor of package managers. Konrad writes:
/usr/bin/python2.1 exists under 7.2, and claims in its startup that it existed in 7.1. You get 1.5.2 if you just type "python", but you get 2.1.1 if you type "python2.1". alias python python2.1... --jh--
Joe Harrington <jh@oobleck.astro.cornell.edu> writes:
RPM. Plus, I can uninstall! What matters is what the general users want, and they have spoken loudly in favor of package managers.
I couldn't agree more. I tend to make many of my own RPMs, and still I prefer RPMs to straightforward installation in /usr/local. Clean updating and uninstalling is a big advantage.
Definitely not in 7.1, there was not Python 2.1 at all (before I installed it by hand). Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
Hello All, Debian (unstable -- which is actually quite stable) provides slightly more up-to-date packages of numeric (20.2) and its extensions. You can find them here: http://packages.debian.org/unstable/math/python-numeric.html http://packages.debian.org/unstable/interpreters/python-numeric-ext.html For a quick hack at a more up-to-date RPM, it _might_ be possible to use alien to convert the *.debs to *.rpms... Scott PS: As an astronomer myself, I am also seeing an increasing interest in my collegues towards python and numeric (especially since I keep preaching the Python gospel to them ;) On January 2, 2002 02:29 pm, Joe Harrington wrote:
-- Scott M. Ransom Address: McGill Univ. Physics Dept. Phone: (514) 398-6492 3600 University St., Rm 338 email: ransom@physics.mcgill.ca Montreal, QC Canada H3A 2T8 GPG Fingerprint: 06A9 9553 78BE 16DB 407B FFCA 9BFA B6FF FFD3 2989
"SR" == Scott Ransom <ransom@physics.mcgill.ca> writes:
SR> Hello All, Debian (unstable -- which is actually quite stable) SR> provides slightly more up-to-date packages of numeric (20.2) SR> and its extensions. You can find them here: SR> http://packages.debian.org/unstable/math/python-numeric.html SR> http://packages.debian.org/unstable/interpreters/python-numeric-ext.html Nothing like 2 days to make it now 20.3 in Debian unstable/Sid. best, -tony -- A.J. Rossini Rsrch. Asst. Prof. of Biostatistics U. of Washington Biostatistics rossini@u.washington.edu FHCRC/SCHARP/HIV Vaccine Trials Net rossini@scharp.org -------------- http://software.biostat.washington.edu/ -------------- FHCRC: M-W: 206-667-7025 (fax=4812)|Voicemail is pretty sketchy/use Email UW: T-Th: 206-543-1044 (fax=3286)|Change last 4 digits of phone to FAX Rosen: (Mullins' Lab) Fridays, and I'm unreachable except by email.
Hi, Try python setup.py bdist_rpm Works like a charm. The same for the subdirectories/subpackages (maybe after installation of the main RPMs). Gerard PS: it is impossible to provide binary RPMs, because there are too many different Linux distributions. On Wednesday 02 January 2002 20:29, Joe Harrington wrote:
Joe Harrington <jh@oobleck.astro.cornell.edu> writes:
If someone can give a precise description of the problem, I'll look at it, I think I have done enough RPMs to find my way... The RPMs generated by Distutils are often good enough, but sometimes need some manual cleanup.
distutils (is distutils a separate thing or part of Python?) at all.
It is a part of Python since Python 1.6.
installation manager. We need these (very) early adopters, so I think that having a current Numeric RPM for i386 Linux (and the equivalent
I agree in principle, but providing Linux binary RPMs is becoming more and more messy: it depends on the Python version and on the distribution, sometimes even the distribution version number. That's a lot of RPMs. Source RPMs are easier, with some care they should work everywhere, but installation requires a compiler plus some of the "-devel" packages that not everyone has.
Agreed. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
In the exchange below, my query is >>, Gerard's response is >, and my present response to that is unquoted.
Yes, mine go into /usr/lib/python2.1
Well, the paths are figured out in /usr/lib/python2.1/distutils/sysconfig.py, using sys.prefix or sys.exec_prefix.
Or that it resides on an NFS mounted volume??? Or some python startup file that messes sys.(exec_)prefix.
Ok, Paul, Konrad, is this what you need?
Every combination of Distro-Version-Python-Version would require its own binary RPM.
If you consider Microsoft Windows one OS, Red Hat Linux another, Debian GNU/Linux a third, and so on, the distinction goes away. The fact that Microsoft has a monopoly is definitely convenient to producers. However, as users, we lose and shouldn't support it.
(B) The other possibility is to include a generic python-numeric.spec file (you build an RPM with rpm -ba python-numeric.spec). I can adapt mine.
Gerard
PS: do you have a RPM based Linux, yourself?
Yes, I run Red Hat. So do the vast majority of US astronomers I have spoken to. I understand that SuSE is equally popular in Europe. A few other dists just copy one of them and add a few packages, and so don't need separate RPMs of their own. If we configure RPMs with the version of python included in these dists, we're not talking about a large number of packages, after all, to get 98% of the users or better. I think you are onto a solution: We build RPMs (and .deb files for Debian, if we think there are enough Debian users to make it worthwhile, and the same for Solaris if it uses a different package manager) for the current release of each popular distribution, with that release's python. That's maybe 4 binary packages per Numeric release (Red Hat, SuSE, Debian, Solaris). We also make a source RPM and distribute with it instructions for building a new binary RPM with appropriately-named version ID (e.g., python-numeric-20.4py2.2glibc6-1). We solicit users of unsupported combinations of OS and Python version to contribute those RPMs along with a simple ASCII form specifying what kind of system made it, contact info, etc. I predict we will not get a huge number of such submissions from outside this list, as the vast majority of Numeric users who upgrade their Python when a new Python release comes out (as opposed to a new OS release) are developers who don't care about the RPM anyway. The ultimate solution is to get this thing incorporated into the python core. --jh--
I can confirm that the Python I use is in /pcmdi/dubois/python. I *never* build into /usr or /usr/local. There are lots of people like me that have multiple pythons around with other stuff added in. I suppose what was bothering people was having to use --prefix because the default will not work because my Python is not in some standard place (said standard place seeming to me to be a mythical location given how I see people actually working). I continue to be less than impressed by RPMs. Every time I try one I get version conflicts with other software.
I *never* build into /usr or /usr/local.
Good! You should leave /usr for your package manager to deal with. However, you should *release* software to install there, which you can with RPM without having to build it in /usr. /usr/local is yours to play with. The only thing in mine is IDL. (...which is why I'd like to delete it!)
There was no documentation anywhere in the download area suggesting the use of --prefix. Even if there were, RPMs get collected and put in archives (rpmfind.net) without any other associated docs. They should always install into the system following the Linux file system standards. Nobody expects to have to configure an RPM. They're just supposed to work, and the vast majority just do.
There are lots of people like me that have multiple pythons around with other stuff added in.
There are lots if *developers* like you. The current group of Numpy users is demographically distinct from the future user base, if Numpy becomes a player for numerical work in science and engineering. We will have many users who have never written a line of C or C++ or FORTRAN and who will run screaming at the mention of the word "tar". We will have high school teachers and students. We will have lots of people who have zero time and zero tolerance for learning unrelated topics like system administration or software tweaking. If we don't care about them, they won't use our software.
I continue to be less than impressed by RPMs. Every time I try one I get version conflicts with other software.
I understand your frustration with the bad RPMs, but it's not the fault of RPM, it's the fault of either the package developer or the person installing the package. If the package developer links against a library that is itself under rapid development, there is a problem waiting to happen. It's worse if that library is included in some or all distributions. The simple solution is to statically link that library, or rename it and include it in the package. If the user has an out-of-date system, it's also trouble. If we build against the C library and python that are in the OS releases, we're in good shape. I don't think we need to bend over backwards to provide RPMs for people who install their own Python, unless a new version of Numeric requires use of an updated Python RPM. Very few (maybe 0.5%?) "mass-market" users will install their own Python. The experts who do can handle a tarball install, or one will build an RPM and distribute it if it's a particularly important release. The conflict problems pale in comparison to the problems with manually building all packages you want to install, particularly if you don't know how to fix makefile problems or don't have a compiler. If you're only installing a few packages, it's not that big a deal. If you're installing 90, it's a month's work by hand or a day with RPM, if you don't know the install procedure for the software beforehand. See ftp://oobleck.astro.cornell.edu/pub/clues.tar.gz for my attempt to tame this problem. As you can see from the dates of the files therein, I abandonned maintaining each package page as it came out under RPM. It was just impossible to keep up. It's trivial under RPM. Plus, I can uninstall! What matters is what the general users want, and they have spoken loudly in favor of package managers. Konrad writes:
/usr/bin/python2.1 exists under 7.2, and claims in its startup that it existed in 7.1. You get 1.5.2 if you just type "python", but you get 2.1.1 if you type "python2.1". alias python python2.1... --jh--
Joe Harrington <jh@oobleck.astro.cornell.edu> writes:
RPM. Plus, I can uninstall! What matters is what the general users want, and they have spoken loudly in favor of package managers.
I couldn't agree more. I tend to make many of my own RPMs, and still I prefer RPMs to straightforward installation in /usr/local. Clean updating and uninstalling is a big advantage.
Definitely not in 7.1, there was not Python 2.1 at all (before I installed it by hand). Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
participants (6)
-
Gerard Vermeulen -
Joe Harrington -
Konrad Hinsen -
Paul F. Dubois -
rossini＠blindglobe.net -
Scott Ransom