[Distutils] Setting environment variables before building eggs w/ extensions using zc.buildout
Nathan R. Yergler
nathan at yergler.net
Wed Mar 14 22:51:46 CET 2007
On 3/8/07, Jim Fulton <jim at zope.com> wrote:
> On Mar 7, 2007, at 2:07 PM, Nathan R. Yergler wrote:
> I meant to reply earlier. Sorry.
> > I had a problem come up today that I've run into before, and I'd like
> > to figure out a way to prevent it from sucking my time down in the
> > future. Several apps we run at Creative Commons use lxml
> > (http://codespeak.net/lxml) for XML, XSLT and XPath processing. lxml
> > builds on libxml2, and provides an element tree interface, plus
> > additional capabilities.
> > We use zc.buildout to assemble our server-side applications, and that
> > has generally been a huge sanity saver. However, lxml can rob some of
> > that sanity periodically. If a system like one of our CentOS machines
> > has an older version of libxml2 installed in /usr/lib, the extension
> > building process will link against it without complaint. That is,
> > until you try to do something added in the later, recommended version
> > of libxml2 like XSLT. Once you scratch your head and remember what
> > the problem is, you remember to export LD_LIBRARY_PATH=/usr/local/lib,
> > and then re-run the buildout.
> > Does zc.buildout currently have any way to set an environment variable
> > during it's run?
> > I didn't see anything at
> > http://cheeseshop.python.org/pypi/zc.recipe.egg/, but I didn't know if
> > there was something at the zc.buildout level (as opposed to the recipe
> > level) that would do the trick. The quick/dirty thing I'd do is
> > export the appropriate value of LD_LIBRARY_PATH.
> I'm a little surprised that this would help at build time. I'm
> surprised that distutils looks at LD_LIBRARY_PATH when deciding where
> to link from.
> Have you looked at:
> > The perhaps more
> > intelligent thing I'd do is use the cmmi recipe to build a version of
> > libxml2 that I *know* has the features I need as part of the buildout,
> > and then point LD_LIBRARY_PATH at that.
> Maybe. That's what I would do. :)
> There are really 2 issues:
> - Whenther to rely on your system libraries,
> - How to tell distutils where the libraries are, both at build and at
> run time.
> The former depends on the library. I've had problems with this
> particular library on Red Hat-based systems, si we typically build
> our own as part of a buildout.
> For the later, the zc.recipe.egg:custom recipe is what we use. It is
> simplest to use the rpath option at build time.
> The rpath option doesn't work for us in production deployments
> because our deployment-install locations are different than our build
> locations, so we use LD_LIBRARY_PATH at run time. Because we use
> zdaemon and zdaemon lets us specify run-time environment variables,
> this isn't a problem. (Note that a Python script can't set it's own
> LD_LIBRARY_PATH, because it is too late -- the library loader has
> already looked for this before a Python script can set it. This
> works with zdaemon because zdaemon creates a separate subprocess for
> the application and the environment variables are set before the
> subprocess runs.)
Ah; luckily we're using zdaemon as well, so I'll go down that path. I
suppose I could combine what Deliverance does with zdaemon and have a
* builds libxml2 and libxslt
* builds lxml against them
* updates my zdaemon.conf to include the resulting lib path
> Jim Fulton mailto:jim at zope.com Python Powered!
> CTO (540) 361-1714 http://www.python.org
> Zope Corporation http://www.zope.com http://www.zope.org
More information about the Distutils-SIG