On Sat, 9 Sep 2000, Greg Ward wrote:
On 05 September 2000, Harry Henry Gebel said:
The contents of the spec file are included in the source RPM, that is why A think that using sys.executable should be the option and not the default.
Dammit, I'm still waiting for a light to flash on so I can understand where you guys are coming from on this. Please indulge me as I think out loud...
Making packages is not that hard. Making GOOD packages is very subtly tricky. My comments below may help your understanding. They may not help find a solution....
Assertions (please tell me if I'm misunderstanding something):
* this isn't really all *that* important, as the build instructions in the .spec file only apply to people building the source RPM -- IOW, the build instructions in the .spec file in no way affect the majority of people who use the RPM, ie. those who just install the "built" RPM
Correct.
* in fact, the only person who *must* build the source RPM is the person who creates the RPM in the first place (although anyone creating binary RPMs for other distributions or other architectures would probably start from the source RPM)
Not exactly. No one ever *must* build a source RPM. If you have a .spec and a source tar/directory you can build a binary RPM without creating a source RPM. Distribution of the binary RPM satisfies 99.99% of the people who want to install an RPM. Those who want to tweak how the binary is created could use the source RPM, but even then (especially with pure python packages) it's not required. I've "rebuilt" RPM's by getting a source tarball and recreating most of the .spec from an rpm -qi query.
I've been thinking hard about this -- hey, my brain is slow this weekend -- and I think I understand it a little better.
Case 1 (the status quo): put "python setup.py ..." in the build/install/ clean instructions in the .spec file. This is bad when the packager, P, uses a "non-standard" python (anything other than /usr/bin/python) to create an RPM that is intended to go in the non-standard location. This is mainly a problem when P immediately creates a built RPM from his new source RPM (the usual case, in fact); if a builder, B, turns source RPM -> built RPM on a separate system, then using the first python in the path -- most likely /usr/bin/python -- might well be the right thing to do.
Close. It really has nothing to do with the source RPM. It's just the usual RPM way of package building. I prefer to "force" my rpm installs (the fake install done by rpm -b during the install step) into a directory OTHER than where it will install on the target machine. Benefits are 1) I can create a binary RPM on a system where I don't hace root access (as if I'd work there...), and 2) you don't step on your production files untill you are satisfied with the RPM. This has the added advantage in that you can now make the RPM relocatable with a little more effort in the .spec file. To make the package relocatable, you capture the library path where you want it to go (presumably the site-packages directory of the default python, but it doesn't have to be) to the location where you just did the fake RPM install. RPM then works it's magic by creating the package from the files in the fake tree so that they will be installed by default into the real site-packages directory. Since the package is relocatable now, the installer could override it to go anywhere. This is really cool, in that now the installer doesn't have to be root. Instead, they can relocate the package anywhere they can write and just add that path to their PYTHONPATH.
Case 2: put sys.executable + " setup.py ..." in the .spec file. This fixes the above problem, but is bad in the case where P accidentally uses a non-standard python to create an RPM that is supposed to go to the standard python installation (/usr). Eg. if I forget that /usr/local/bin/python is first in my path, then any source RPMs I create will refer to /usr/local/bin/python in the .spec file, and building those source RPMs will either fail (on systems that don't have /usr/local/bin/python, probably the vast majority of installed Linux boxen out there) or will generate an RPM with the "non-standard" destination of /usr/local.
There's a little more confusion here, and I'm not exactly sure how it works with bdist_rpm. RPM does some additional hacking beyond what's in the .spec file. It looks especially for #! magic in package files but also for referenced libraries and anything else it thinks the package is going to need. RPM then (theoritcally, in a helpful fashion) adds these things to the "Requires:" tag in the binary package. SO, if it identifies /usr/local/bin/python as a requirement for the package, it will not install (without --force) on a target system that does not have /usr/local/bin/python. (Yes, I have done this...the original pygresql package went out this way...sigh). I don't think RPM deduces requires from processors used in the setup, config, build, install steps, but if the packager is using a /usr/local/bin python, it's very likely to be references somewhere in the package, too.
In the latter case -- accidentally using the wrong python -- it would be best if the Distutil issued a warning. I don't see an easy way to detect this, especially if someone is deliberately creating an RPM to install modules to a non-standard location. (Eg. Andrew's case of having python in /www/python/bin/python: he might want to make RPMs of common modules to install on all the web developers' workstations in /www/python.)
This is why I'm struggling with pkgtool (and even worse, sdux). If the bdist_whatever's make the package relocatable, it doesn't matter what python was used in the build (location-wise anyway). Then the issue shift's to installation time. How do you determin the TARGET system's "prefered" python? This could be retrieved either by executing python or querying RPM in a preinstall step, then automatically reloacting the package where it should go. Except you also have to be prepared to skip the auto-relocation if the installer is relocating it with RPM options.
Both situations are subtle errors on the packager's part, and neither seem to have obvious automatic solutions. The "fix" is social engineering: let the packager decide what he wants to do with options to the bdist_rpm command, and make sure the rationale for each option is carefully documented.
Like I said...Packaging is easy. Making easy packages are hard. I'd vote for making bdist_rpm make things relocatable with a default pre-install to do auto-relocation IF I still had any sanity from actually trying to do this with other package types. Maybe I'm too anal about my packages, but good ones make it sooooo much easier to administer large number of machines.
The question is: which failure is more obvious and happens closest to the packager, the one who made the mistake (failed to RTFM, etc.)? That one should be the default. It seems to me that creating a source RPM (spec file) that immediately fails because of an explicit /path/to/python is better than one that works, but possibly wrongly, because of implicitly using the first python on the path.
Agreed, except that the source RPM is still only going to be used by people who are already prepared to get in and hack at it anyway.
IOW, "Explicit is better than implicit", even in snippets of shell code included in a .spec file bundled in a source RPM.
Hey, you're not trying to suggest that maybe, possible, ONE thing from bdist_pkgtool was almost, kinda, maybe on the right track ;-) Anyway I said this probably wouldn't be a solution, but I hope I clarified the problem al little.... mwa