[SciPy-user] bdist_rpm build error in scipy_core from CVS

Fri Jan 28 11:42:06 EST 2005

[sorry if this is repeated, but as far as I can tell it never reached the 
list, so I'm trying a resend]

Joe Cooper wrote:
> Hi all,
> 
> There's something gone awry in recent scipy_core CVS.  When I last 
> packaged it up, it worked OK with bdist_rpm, but now it fails with the 
> following traceback:

[...]

I actually don't have a solution to this, the last version I built an RPM for
was CVS from a few weeks ago:

In [2]: scipy.__version__
Out[2]: '0.3.2_300.4526'

It worked fine.  But I want to mention something in this thread, b/c it's RPM
related.  If anyone is going to have a look at the scipy rpm build process, it
might be worth fixing a few (very minor) issues.  I had also promised to pass
some of this info to others (esp. S. Walton), so here it goes.

I recently set up a group of Fedora3 repositories for our systems, with
multiple architectures and therefore different ATLAS releases (which implies
different Numeric/Scipy RPMs as well).  For the most part the process went
quite well, I finally got it down to the following:

cd /path/to/scipy/source/dir
cd scipy_core
pybrpm-arch Scipy_core
cd ..
pybrpm-arch SciPy

where pybrpm-arch is a simple shell script (attached) which automates the
building and installation of an rpm via bdist_rpm.  But the packages had to be
split in Scipy_core and SciPy (with that funny capitalization), otherwise
things won't install, since Scipy_core contains the critical scipy_distutils.

I'm not sure where the discussion stands on what to include into scipy_core
and what should go in the main package.  Currently, scipy_core includes:

/scipy_base
/scipy_distutils
/scipy_test
/weave

and SciPy contains

/gui_thread
/scipy

(all of this in /usr/lib/python2.3/site-packages).  I don't have a problem
with that, I just list it here for reference.

The only thing I'd like to see fixed is the awkward naming of the packages:
why not be consistent and just name the two proper RPMs

scipy_core
scipy

This post is mainly to just ask for this minor change, and provide the install
scripts which may be of use to others for managing Fedora-based environments.
   They all rely on a few environment variables which need to be correctly
configured.  This is from my root/.tcshrc:

##############################################################################
#
# Local yum configuration
#
alias yinst 'yum -y install'
alias yup   'yum -y update'

# Fedora Release version, as seen by YUM.
setenv RELEASEVER 3
# Architecture-specific flag, as per the ATLAS binaries convention
setenv ARCH P4SSE2

# Yum allows expanding $YUM0-9 variables in its .conf files, so by setting
# this variable we can manage architecture-specific repositories.

setenv YUM0 /usr/local/installers/yum/fc${RELEASEVER}
setenv YUM1 $ARCH
setenv YUM2 ${YUM0}-arch/${YUM1}
##############################################################################

My local yum repo is organized as:

root at planck[yum]# d
/usr/local/installers/yum
total 20
drwxr-xr-x  3 root 4096 Jan 27 15:47 fc3/
drwxr-xr-x  5 root 4096 Jan  7 12:00 fc3-arch/
drwxr-xr-x  3 root 4096 Jan 19 18:28 fc3-shared/
-rw-r--r--  1 root  984 Jan 10 16:24 README
-rwxr-xr-x  1 root   95 Jan  4 11:53 update-repo*

where fc3-arch contains the architecture-specific repositories:

root at planck[fc3-arch]# d
/usr/local/installers/yum/fc3-arch
total 12
drwxr-xr-x  3 root 4096 Jan 25 12:34 P4SSE2/
drwxr-xr-x  3 root 4096 Jan 19 18:29 P4SSE2_2HT/
drwxr-xr-x  3 root 4096 Jan 19 18:29 PIIISSE1/

and update-repo is a trivial one-liner I run whenever any repo changes (it's
symlinked from all of them):

root at planck[yum]# cat update-repo
#!/bin/sh
# as of FedoraCore3, this is the proper command to update a repository

createrepo .

#########################

This setup allows me to manage multiple heterogeneous machines which fetch
their updates from a single NFS-shared group of yum repos, without any
conflicts and using the same config files (in /etc/yum.repos.d) for all.  Here
are the relevant repo files:

root at planck[yum.repos.d]# cat local.repo
### Local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

[local]
name = Local packages (Fedora Core $releasever)
baseurl = file://$YUM0
enabled=1

#########################
root at planck[yum.repos.d]# cat local-arch.repo
### Architecture-dependent local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

[local-arch]
name = Architecture-dependent local packages (Arch: $YUM1)
baseurl = file://$YUM2
enabled=1

#########################
root at planck[yum.repos.d]# cat local-shared.repo
### Architecture-dependent local packages

# These are directories with locally available rpms.  Remember to run
# 'createrepo .' when new rpms get added, so that the repodata/ subdir needed
# by yum is updated.

# The -shared repo contain RPMs which were written to go into /usr/local/, and
# hence are effectively shared via NFS.  This repository should only be
# enabled on the host which hosts /usr/local.

[local-shared]
name = Shared local packages
baseurl = file://$YUM0-shared
enabled=1

####################

I've written also some code to auto-generate architecture-specific RPMs out of
the scipy.org ATLAS binary tarballs, which I can provide if anyone is interested.

All this, combined with some more scripts to auto-generate kickstart install
scripts, gives a pretty reasonable solution for managing the problem of
multiple machines, with multiple architectures and in-house managed RPMs for
rapidly changing code (like scipy, ipython and matplotlib), while keeeping my
sanity and time to do my official job (research).

We can now go from blank hard disk to fully updated box in about 1 hour, with
about 3 minutes of human intervention.  And the machines stay up to date via
yum, including regarding our own managed packages.

Hopefully some of this is useful to others.

Cheers,

f.

ps. And Joe, good luck with your bug :)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpm-arch
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050128/fcc2ac14/attachment.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpminst
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050128/fcc2ac14/attachment-0001.ksh>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pybrpm-noarch
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20050128/fcc2ac14/attachment-0002.ksh>