[Distutils] python eggs and lxml
Phillip J. Eby
pje at telecommunity.com
Sat Sep 24 17:23:52 CEST 2005
At 11:13 AM 9/24/2005 +0200, Martijn Faassen wrote:
>I guess for Windows, I'd have make distutils run the configure script,
>then extract the dlls it produces and stuff them in the egg somehow. Any
>direction you'd point me towards for this?
You'll need to have your setup script determine programmatically that it's
running on Windows, and then add data files and 'eager_resources' to the
project accordingly. In other words, to include shared libraries that
aren't distutils-built extensions, you have to treat them as data files,
but you must also list them in the 'eager_resources' setup keyword, as
so that the shared libraries will be automatically extracted to disk before
they're linked to by any C extensions.
You can do this on Unix platforms, too, of course, but the filenames will
naturally be different.
>In this case, on Linux, I'd want to run the configure script when the egg
>is installed instead of when it's created, and stuff the .so files in the
>same place the egg is being installed to.
Eggs are not source distributions - they're prebuilt binaries for a
particular platform. EasyInstall will find and use source distributions if
there's no binary for the system, so from a user point of view it certainly
happens when the egg is "installed", but that's only because the actual egg
is being *built* locally.
So, in any situation where you're not using the platform libraries, you'll
need to follow the same embedding steps as for Windows - i.e., build the
libraries and include them as data files.
If you're linking with the platform libraries, you don't need to include
them as data files.
It's best, however, if you not think of this as install vs. create. The
only installation steps eggs have are:
* adding them to sys.path
* creating local wrappers to run programs contained in them
Everything else occurs when the egg is built, it's just that on Unix-y
platforms you're more likely to be building the egg locally from source,
rather than downloading a pre-built egg, if it contains C extensions. For
Python-only projects, of course, eggs are cross-platform and ready-to-use
>>Not all libraries can be bundled by source, of course. Sometimes you
>>really need to use whatever the "system version" is, for one reason or
>>another. Database clients, for example, are something you really really
>>want to use the local version for.
>Right, there are competing use cases here. What I'd like is an easy
>install for lxml that just works for people, without them having to worry
>about the right lxml2 versions being installed, etc. On Windows this means
>binaries, and on Linux this likely means it'll just compile upon install.
Or more precisely, it means compiling on egg build, regardless of
platform. It's just that for some platforms, you'll build the egg and
distribute it instead of the end user building their own.
Probably the simplest way to do this is to subclass setuptools 'build_ext'
command and extend run() to 'configure' and 'make' the libraries before
>Some classes of people, like distributors and some sysadmins, care about
>using the platform version of libxml2, and I'd also want to create an egg
>that allows you to install against the platform libraries. Would this be
>possible to be the same egg or would a different egg be needed? If a
>different egg, how does this work with the dependency system? I.e. these
>two eggs would be alternatives of each other dependency-wise.
An egg is not a project, or vice versa. What you mean is that you want
your project to be able to be built against different targets. The eggs
are what gets built. If you've only used EasyInstall as a user, not a
developer, this line is more blurry because EasyInstall downloads projects
and builds the eggs for you if the author did not provide any eggs usable
on your platform. But there is a distinction, and on the authoring side
you should focus on the project and its ability to be built for different
targets. Providing actual eggs is a convenience for your users, and does
not enter into your development process. It's strictly a
Anyway, since the audience that wants to do this are more advanced users,
and probably building your project individually rather than as part of a
multi-project EasyInstall, I think it's reasonable to just have some value
in a configuration file or environment variable or the standard distutils
setup.cfg that allows skipping the configure+make. For that matter, just
documenting how to patch the setup script to use the platform libraries
might be sufficient for that audience in the case of this kind of embedding.
More information about the Distutils-SIG