Thanks to Harry Henry Gebels bdist_rpm module, I hacked out a bdist_pkgtool module to create binary packages for those of us stuck with Solaris. The module is attached. As a first pass, it takes somewhat of a sledgehammer approach, but it does work for at least a simple pure-python module. I have not tested pre/post-install scripts, and I doubt if they'd work anyway. I did however, include a "request" script which searches for the python installation directory before the package is installed and automatically relocates the module to the site-packages directory on the installation target. The same module package can be installed on machines with python installed in different locations and it will figure out the correct place to go. Let me know of any problems. I'm off to do HP-UX depot format now..... Mark Alexander mwa@gate.net
On 24 August 2000, Mark W. Alexander said:
As a first pass, it takes somewhat of a sledgehammer approach, but it does work for at least a simple pure-python module.
"Sledgehammer" is one way to put it. It might be a more convincing translation if you finished the s/rpm/pkgtool/ job. ;-) The major problems I see with this adaptation are: * reuse by cut and paste doesn't count. I'm going to stop accepting patches that just copy an existing module and hack into shape... *now*. If you want to implement command bdist_foo, and it's very similar to bdist_bar, then refactor first! * it shouldn't be the job of a shell script generated by bdist_pkgtool.py to go from sys.prefix to <prefix>/lib/pythonx.y/site-packages; that is already done by the 'install' command when run with a certain prefix. Or you can use the 'get_python_lib()' function from distutils.sysconfig. Two places with this logic are enough... Oh, and of course I have a burning and irrational hatred of Solaris which, unlike my burning and irrational hatred of Windows, stems from actual use. So the idea of lifting a finger to help Sun, even as indirectly as this, really sticks in my craw. >grin< Greg -- Greg Ward - Unix nerd gward@python.net http://starship.python.net/~gward/ Pointers are Arrays; Code is Data; Time is Money
On Sat, 26 Aug 2000, Greg Ward wrote:
On 24 August 2000, Mark W. Alexander said:
As a first pass, it takes somewhat of a sledgehammer approach, but it does work for at least a simple pure-python module.
"Sledgehammer" is one way to put it. It might be a more convincing translation if you finished the s/rpm/pkgtool/ job. ;-)
Told ya so! I was only trying to convince myself that I could do it without disrupting my production schedule. It's a testimony to Mr. Gebel's rpm module (sp?) that I got it to work without knowing poot about distutils. And by finish you mean "finish destroying Mr. Gebel's code"? I don't think so....When I get back to it, it will be to do it right.
The major problems I see with this adaptation are:
* reuse by cut and paste doesn't count. I'm going to stop accepting patches that just copy an existing module and hack into shape... *now*. If you want to implement command bdist_foo, and it's very similar to bdist_bar, then refactor first!
No problem. For me, this was jump in, get my feet wet and see if I could make it work. bdist_sdux is better in that at least I understood the steps. My first priority is to package a bunch of local mods, and this has got that licked. I'm more than happy to take advice on the distutils way and improve it as time permits.
* it shouldn't be the job of a shell script generated by bdist_pkgtool.py to go from sys.prefix to <prefix>/lib/pythonx.y/site-packages; that is already done by the 'install' command when run with a certain prefix. Or you can use the 'get_python_lib()' function from distutils.sysconfig. Two places with this logic are enough...
The problem with that is that you either A) get the info at build time or B) run distutils to do the actual binary install on the target. In my case, A is not a reliable indicator of where python is installed on the target machine. B loses the advantage of the native packager in that the file inventory is not registered. The script you're refering to is created to be placed into the package to be run at "request" time, where packages can ask the admin what to do. Instead of asking, the script figures it out by itself. This means that with one pkgadd, I can install any number of modules with no further intervention. Anyway, Using the standard package install scripts to find python and automatically relocate is best of several poor solutions. If you've got a better idea, I'll take it.
Oh, and of course I have a burning and irrational hatred of Solaris which, unlike my burning and irrational hatred of Windows, stems from actual use. So the idea of lifting a finger to help Sun, even as indirectly as this, really sticks in my craw. >grin<
There's nothing irrational about it;-) But, distutils really needs to support it, along with the other big unnamed ones. Having to work on Solaris and HP are bad enough, but to do so without Python is simply cruel and unusual punishment. And, since most my sysadmin scripts are in Python, being able to effortlessly maintain package modules for both is why I need distutils. If it was just rpm, I could manage without. So...don't help Sun, help me and others like me. First, forget this one, I'll clean it up later. Give me constuctive criticism on bdist_sdux (which has a completely different relocation nightmare) and I'll retrofit the advice to pgktool. At the very least, point me to some description of the inputs to the bdist_XXX modules and the setup and use of bdist module options. I mean, if you want quality bdist_modules, you need to give quality instructions. And I DON'T mean now as in "drop everything, this is important, I gotta get it done", either, cause the time I have to devote is pretty slim anyway. That's the biggest reason I wanted to post something, so that maybe someone else would be interested. That, and I think I promised to do it sometime last year(?)... Unfortunately, from what I've seen on this list I'm the lonely Solaris/HP repairman. So, what I did will hold me for a while. And finally, the reason I'm suddenly pushed to get things packaged and out the door is because I've finally got some help to start dumping on^W^W training. I would like to note for the record, that without distutils I'd have spent at least days, probably weeks, getting things organized, packaged, and prepared for moving to 1.6 on all my platforms. Your work is really paying off, and it is most sincerely appreciated! Mark Alexander mwa@gate.net
On 27 August 2000, Mark W. Alexander said:
And by finish you mean "finish destroying Mr. Gebel's code"? I don't think so....When I get back to it, it will be to do it right. [...] No problem. For me, this was jump in, get my feet wet and see if I could make it work. bdist_sdux is better in that at least I understood the steps. My first priority is to package a bunch of local mods, and this has got that licked. I'm more than happy to take advice on the distutils way and improve it as time permits.
OK, good, one more budding Distutils hacker with feet wet. Excellent! Need all the help we can get. I will take a closer look at bdist_sdux; I was worried that it would be a hatchet job similar to bdist_pkgtool, and I hate reading cut 'n pasted code. But first, you have to tell me what "sdux" stands for. I keep mentally dropping the "d"...
The problem with that is that you either A) get the info at build time or B) run distutils to do the actual binary install on the target. In my case, A is not a reliable indicator of where python is installed on the target machine. B loses the advantage of the native packager in that the file inventory is not registered.
Ahh, gotcha. Idea: don't use Distutils to do the install, but do take advantage of it for the install-time scripts. Or rather, take advantage of Python and assume that the Distutils are installed on the target machine. Then you can do from distutils.sysconfig import get_python_inc libdir = get_python_lib(plat_specific=1) ... instead of that nasty shell trickery-pokery. Of course, you may have a slight chicken/egg problem in getting the Distutils installed, if these target platforms are running 1.5.2. Umm, left as an exercise for the reader... ;-) Note that the other Unix bdist_* commands (dumb and rpm, at this date) don't do anything smart about install location. They just mimic whatever the structure on the build host was. This is just fine for bdist_rpm; if you're building and RPM for Red Hat 6.2, you should be doing so with the Red Hat 6.2 Python interpreter. For bdist_dumb, it's just laziness on my part -- and the only real way to deal with the potential prefix/exec-prefix distinction. General principle: when writing a bdist_* command, I won't bite your head off if you assume that the target system's Python installation is the same as the build system's. At least for Unix: I think on Windows, it's important to adjust to the user's preferred location for Python. Probably the same for Mac OS... someday. ;-)
Oh, and of course I have a burning and irrational hatred of Solaris which, unlike my burning and irrational hatred of Windows, stems from actual use. So the idea of lifting a finger to help Sun, even as indirectly as this, really sticks in my craw. >grin<
There's nothing irrational about it;-) But, distutils really needs to support it, along with the other big unnamed ones. Having to work on Solaris and HP are bad enough, but to do so without Python is simply cruel and unusual punishment.
Yeah, you're right; no point in punishing the poor hapless souls stuck with the evil spawn of Sun. My sympathies.
So...don't help Sun, help me and others like me. First, forget this one, I'll clean it up later. Give me constuctive criticism on bdist_sdux (which has a completely different relocation nightmare) and I'll retrofit the advice to pgktool. At the very least, point me to some description of the inputs to the bdist_XXX modules and the setup and use of bdist module options. I mean, if you want quality bdist_modules, you need to give quality instructions.
What, the source code isn't documentation enough? ;-) Well, Fred has been pushing me to write some "Distutils Internals" documentation. Back to the docs... Greg -- Greg Ward - Unix geek gward@python.net http://starship.python.net/~gward/ Very few profundities can be expressed in less than 80 characters.
On Mon, 28 Aug 2000, Greg Ward wrote:
OK, good, one more budding Distutils hacker with feet wet. Excellent! Need all the help we can get.
I will take a closer look at bdist_sdux; I was worried that it would be a hatchet job similar to bdist_pkgtool, and I hate reading cut 'n pasted code. But first, you have to tell me what "sdux" stands for. I keep mentally dropping the "d"...
SD-UX is Software Distributor for HP-UX. I have the same problem with the 'd'. I'm not attached to the name, so _hpux or _depot (that's what an hp package is called) or even just _hp. I'm just not positive that HP is the only one that uses this format.
Ahh, gotcha. Idea: don't use Distutils to do the install, but do take advantage of it for the install-time scripts. Or rather, take advantage of Python and assume that the Distutils are installed on the target machine. Then you can do
from distutils.sysconfig import get_python_inc
libdir = get_python_lib(plat_specific=1) ...
instead of that nasty shell trickery-pokery.
Hmm, so I would then invoke distutils 'install' and force it to that directory? Then I'd either have to force the file inventory somehow. I'm not aware of a clean way to do that, although the inventory files are text that could be manipulated. I tend to favor letting the native tool do as much as possible, though. My assumption is that if you're building binaries, your doing it for people who don't care about distutils or anything else about how the binary was created. They just want it to work like everything else they do. Maybe I've just been around too many people who prefer ignorance....
Of course, you may have a slight chicken/egg problem in getting the Distutils installed, if these target platforms are running 1.5.2. Umm, left as an exercise for the reader... ;-)
I can cope. The package can be made dependent on python 1.6. If necessary, I can provide python binary packages for Solaris and HP and submit them to the usual sites. As long as the package criteria is known, making dependencies is a breeze.
Note that the other Unix bdist_* commands (dumb and rpm, at this date) don't do anything smart about install location. They just mimic whatever the structure on the build host was. This is just fine for bdist_rpm; if you're building and RPM for Red Hat 6.2, you should be doing so with the Red Hat 6.2 Python interpreter. For bdist_dumb, it's just laziness on my part -- and the only real way to deal with the potential prefix/exec-prefix distinction.
General principle: when writing a bdist_* command, I won't bite your head off if you assume that the target system's Python installation is the same as the build system's. At least for Unix: I think on Windows, it's important to adjust to the user's preferred location for Python. Probably the same for Mac OS... someday. ;-)
This assumption is too painfull for me. When I assumed my current position, I took over dozens of archaic machines where the disks where sliced horribly. Of course, I HAD to have python everywhere, so I built packages that relocated (and linked to /usr/bin and /usr/local/bin). Because of that experience, I've become a fanatic advocate of relocatable packages. If the installer doesn't care, then default directories are fine. If they have any need to be creative (like building an NFS utility partition) I don't feel it's a packager's place to cripple them. Making packages relocatable is not that difficult (even though making packages that make relocatable packages can twist your brain....).
Yeah, you're right; no point in punishing the poor hapless souls stuck with the evil spawn of Sun. My sympathies.
Thanx B-\
What, the source code isn't documentation enough? ;-)
Usually, yes. I must have looked at the Command class for at least 5 minutes before trying sdux! Time is just too short for me right now.
Well, Fred has been pushing me to write some "Distutils Internals" documentation. Back to the docs...
I did see you mention somewhere that there was some docs in CVS. I have had problems getting the CVS version since I'm behind a firewall. If there's something there that's helpfull, I'd appreciate it included in the next snapshot (or even a seperate doc-only snapshot). But rather than "Distutils Internals", may I recommend a "command writer's guide" first? My impression is that Distutils is both robust and enough for 1.0 and that those already involved are probably knowledgable enough to squash bugs. The biggest are for improvement now lies in additional package format support. And let's face it, making packages ain't brain surgery. If I can make a bdist_pkgtool_sux_but_works in a couple hours by stepping through someone elses bdist and typing over lines, it can't require that much knowledge of distutils internals. Actually, I'm now just sorry I didn't make the attempt a long time ago. Once I get the hang of it, I'd even be willing to install debian somewhere to try _deb if no one else has committed to it. (But do NOT ask me to do AIX!) mwa
participants (2)
-
Greg Ward
-
Mark W. Alexander