[Distutils] Setting environment variables before building eggs w/ extensions using zc.buildout
jim at zope.com
Thu Mar 8 16:13:51 CET 2007
On Mar 7, 2007, at 2:07 PM, Nathan R. Yergler wrote:
I meant to reply earlier. Sorry.
> I had a problem come up today that I've run into before, and I'd like
> to figure out a way to prevent it from sucking my time down in the
> future. Several apps we run at Creative Commons use lxml
> (http://codespeak.net/lxml) for XML, XSLT and XPath processing. lxml
> builds on libxml2, and provides an element tree interface, plus
> additional capabilities.
> We use zc.buildout to assemble our server-side applications, and that
> has generally been a huge sanity saver. However, lxml can rob some of
> that sanity periodically. If a system like one of our CentOS machines
> has an older version of libxml2 installed in /usr/lib, the extension
> building process will link against it without complaint. That is,
> until you try to do something added in the later, recommended version
> of libxml2 like XSLT. Once you scratch your head and remember what
> the problem is, you remember to export LD_LIBRARY_PATH=/usr/local/lib,
> and then re-run the buildout.
> Does zc.buildout currently have any way to set an environment variable
> during it's run?
> I didn't see anything at
> http://cheeseshop.python.org/pypi/zc.recipe.egg/, but I didn't know if
> there was something at the zc.buildout level (as opposed to the recipe
> level) that would do the trick. The quick/dirty thing I'd do is
> export the appropriate value of LD_LIBRARY_PATH.
I'm a little surprised that this would help at build time. I'm
surprised that distutils looks at LD_LIBRARY_PATH when deciding where
to link from.
Have you looked at:
> The perhaps more
> intelligent thing I'd do is use the cmmi recipe to build a version of
> libxml2 that I *know* has the features I need as part of the buildout,
> and then point LD_LIBRARY_PATH at that.
Maybe. That's what I would do. :)
There are really 2 issues:
- Whenther to rely on your system libraries,
- How to tell distutils where the libraries are, both at build and at
The former depends on the library. I've had problems with this
particular library on Red Hat-based systems, si we typically build
our own as part of a buildout.
For the later, the zc.recipe.egg:custom recipe is what we use. It is
simplest to use the rpath option at build time.
The rpath option doesn't work for us in production deployments
because our deployment-install locations are different than our build
locations, so we use LD_LIBRARY_PATH at run time. Because we use
zdaemon and zdaemon lets us specify run-time environment variables,
this isn't a problem. (Note that a Python script can't set it's own
LD_LIBRARY_PATH, because it is too late -- the library loader has
already looked for this before a Python script can set it. This
works with zdaemon because zdaemon creates a separate subprocess for
the application and the environment variables are set before the
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Distutils-SIG