Question about easy-install.pth
I am having troubles working with setuptools on Solaris. The Solaris operating system normally installs modules as packages which contain binaries. This is unlike other Linux operating systems where, for exmaple, an RPM would download the source and build it on the user's machine when they install the RPM. So, to create packages on Solaris we normally install a module to a temporary directory such as /var/tmp/pkgbuild-foo/usr, package up the files that are built, and when the user installs the package these files are then installed to their system. However, we are having problems with figuring out how to properly create the /usr/lib/python2.4/site-packages/easy-install.pth file using our build system. What happens is that each package ends up with its own easy-install.pth file which only contains the information for that one module. Users can't install two such packages at the same time because it creates a file conflict, two packages can't install the same file. Solaris packages do have pre-install, post-install, pre-uninstall and post-uninstall scripts so that we could do something like avoid installing the file as a part of the package and instead generate it on-the-fly via scripting. However, it doesn't seem that setuptools provides any mechanisms for doing this easily. Though I'm not very familiar with setuptools or easy_install, so I hope that I'm missing something. Aside from writing our own code to manually manage adding and removing entries to/from this file, is there any way that setuptools allows you to manage this file when installing binaries to a system, or does setuptools assume the file only needs to be managed when you build a module? I'm hoping to avoid some hacky solution where we try and hack the easy-install.pth file by hand when users install or uninstall packages. Since users can install or uninstall any random combination of packages which may need an easy-install.pth file, I can imagine that it would get complicated to properly manage it via our own scripts. Or is there some other solution that might be useful in our situation that might avoid the need for multiple packages to need this file, or might it be possible for different modules to use differently named files instead of a commonly named easy-install.pth file. Brian
On Wed, Oct 1, 2008 at 2:12 AM, Brian Cameron
I am having troubles working with setuptools on Solaris. The Solaris operating system normally installs modules as packages which contain binaries. This is unlike other Linux operating systems where, for exmaple, an RPM would download the source and build it on the user's machine when they install the RPM.
Actually RPMs usually are "binary". You may be thinking SRPM files (source RPM). Also, not all Linux's use RPM.
So, to create packages on Solaris we normally install a module to a temporary directory such as /var/tmp/pkgbuild-foo/usr, package up the files that are built, and when the user installs the package these files are then installed to their system.
What you'll want to do is to package up the files by creating an egg archive instead. In its simplest form and egg file is basically just a zip file which contains all the python files as well as a small amount of meta data. You use setuptools to create these egg archives. Note that it may be easier to think of an egg file as a "binary" package, much like a tar file, and not as a source package. The best way to make an egg file is to first create a setup.py file which describes the python module you're packaging along with a bit of meta data. At the bare minimum it may be just: #!/usr/bin/env python from setuptools import setup setup( name="mymodule", version="1.0", description="My python module" ) although you'll probably want to add more meta data to it. Then you create an egg file for it using something like: python setup.py bdist_egg -d /tmp/eggfiles and you'll get a file like /tmp/eggfiles/mymodule-1.0-py2.4.egg
However, we are having problems with figuring out how to properly create the /usr/lib/python2.4/site-packages/easy-install.pth file using our build system. What happens is that each package ends up with its own easy-install.pth file ...
The easy-install.pth file should not be part of the "package" (and it won't be if you use egg files as your package). The easy-install.pth file is instead created or modified as a side effect of installing your package. The easy_install command does all the hard work for you.
Aside from writing our own code to manually manage adding and removing entries to/from this file, is there any way that setuptools allows you to manage this file when installing binaries to a system...
I'm hoping to avoid some hacky solution where we try and hack the easy-install.pth file...
You definitely want to avoid hacking the easy-install.pth file if possible. It's format could potentially change some day. Also it can be tricky if you ever deal with a multiple version installation. Fortunately the easy_install command handles that file for you. Basically you create your "binary" package as an egg file, and then when installing your package you invoke easy_install to copy the egg file into the proper place and register it in the easy-install.pth file. If you have your egg file(s) in the local filesystem, you would typically install/upgrade it like: easy_install -U some-package.egg where -U says to install or upgrade if an older version was already present. Depending on how standalone you want it, you may also want to include the option, -H "" , which prevents easy_install from searching the net for the egg file. For completeness I should note that you can opt not to use setuptools or even distutils at all. You could then just bundle all your module files up as an ordinary tar or cpio archive and then later install it by just unarchiving them into the site-packages directory on a different server. You could also create a *.pth file if you need to (named after your module like "mymodule.pth"; not easy-install.pth). However you're on your own in terms of any sort of dependency processing or version conflicts. This is though probably the closest to your original intent of just wanting to archiving files post-install, and extracting them as-is on another server. -- Deron Meranda
At 01:12 AM 10/1/2008 -0500, Brian Cameron wrote:
I am having troubles working with setuptools on Solaris. The Solaris operating system normally installs modules as packages which contain binaries. This is unlike other Linux operating systems where, for exmaple, an RPM would download the source and build it on the user's machine when they install the RPM.
So, to create packages on Solaris we normally install a module to a temporary directory such as /var/tmp/pkgbuild-foo/usr, package up the files that are built, and when the user installs the package these files are then installed to their system.
When you do that, use "setup.py install --root=/var/tmp/pkgbuild-foo" to install the package, and you'll be good to go. That will automatically tell setuptools to install the package in a way that supports being installed like this; any .pth files generated will be named uniquely for that package, rather than using easy-install.pth. In fact, most packages won't generate a .pth file at all when installed this way. Note, too, that the same --root option is also safe for installing plain distutils packages; it's just that setuptools also uses it to enable a distutils-compatible (and packaging tool compatible) installation mode.
Phillip & Others: Thanks so much for taking the time to answer my questions, I am a newbie to setuptools so I apologize if I was just being thick. Phillip, your suggested solution of using --root solved my problem. I was previously using --prefix instead, but using --root creates the packages more in the way that work well with Solaris packaging. Brian
When you do that, use "setup.py install --root=/var/tmp/pkgbuild-foo" to install the package, and you'll be good to go. That will automatically tell setuptools to install the package in a way that supports being installed like this; any .pth files generated will be named uniquely for that package, rather than using easy-install.pth. In fact, most packages won't generate a .pth file at all when installed this way.
Note, too, that the same --root option is also safe for installing plain distutils packages; it's just that setuptools also uses it to enable a distutils-compatible (and packaging tool compatible) installation mode.
Brian Cameron wrote:
You're actually describing pretty much exactly how rpm works :-) As such, the Fedora Project has had to deal with many of the same issues in packaging python packages as you're running into. This pairs of pages may help you: http://fedoraproject.org/wiki/Packaging/Python http://fedoraproject.org/wiki/Packaging/Python/Eggs Note that we're only one Linux distribution. Other Linux distributions have had to tackle these issues in similar ways. I know that Debian, for instance, also has a comprehensive set of Guidelines which includes useful information on packaging python modules. -Toshio
participants (4)
-
Brian Cameron -
Deron Meranda -
Phillip J. Eby -
Toshio Kuratomi