[Distutils] Setting environment variables before building eggs w/ extensions using zc.buildout

[Nathan R. Yergler]

Late reply...

On 3/8/07, Jim Fulton <jim at zope.com> wrote:
>
> On Mar 7, 2007, at 2:07 PM, Nathan R. Yergler wrote:
>
> I meant to reply earlier. Sorry.
>
> > I had a problem come up today that I've run into before, and I'd like
> > to figure out a way to prevent it from sucking my time down in the
> > future.  Several apps we run at Creative Commons use lxml
> > (http://codespeak.net/lxml) for XML, XSLT and XPath processing.  lxml
> > builds on libxml2, and provides an element tree interface, plus
> > additional capabilities.
> >
> > We use zc.buildout to assemble our server-side applications, and that
> > has generally been a huge sanity saver.  However, lxml can rob some of
> > that sanity periodically.  If a system like one of our CentOS machines
> > has an older version of libxml2 installed in /usr/lib, the extension
> > building process will link against it without complaint.  That is,
> > until you try to do something added in the later, recommended version
> > of libxml2 like XSLT.  Once you scratch your head and remember what
> > the problem is, you remember to export LD_LIBRARY_PATH=/usr/local/lib,
> > and then re-run the buildout.
> >
> > Does zc.buildout currently have any way to set an environment variable
> > during it's run?
>
> No.
>
> >   I didn't see anything at
> > http://cheeseshop.python.org/pypi/zc.recipe.egg/, but I didn't know if
> > there was something at the zc.buildout level (as opposed to the recipe
> > level) that would do the trick.  The quick/dirty thing I'd do is
> > export the appropriate value of LD_LIBRARY_PATH.
>
> I'm a little surprised that this would help at build time.  I'm
> surprised that distutils looks at LD_LIBRARY_PATH when deciding where
> to link from.
>
> Have you looked at:
>
>    http://www.python.org/pypi/zc.recipe.egg#creating-eggs-with-
> extensions-neededing-custom-build-settings
>
> >   The perhaps more
> > intelligent thing I'd do is use the cmmi recipe to build a version of
> > libxml2 that I *know* has the features I need as part of the buildout,
> > and then point LD_LIBRARY_PATH at that.
>
> Maybe. That's what I would do. :)
>
> There are really 2 issues:
>
> - Whenther to rely on your system libraries,
>
> - How to tell distutils where the libraries are, both at build and at
> run time.
>
> The former depends on the library.  I've had problems with this
> particular library on Red Hat-based systems, si we typically build
> our own as part of a buildout.
>
> For the later, the zc.recipe.egg:custom recipe is what we use.  It is
> simplest to use the rpath option at build time.
>
> The rpath option doesn't work for us in production deployments
> because our deployment-install locations are different than our build
> locations, so we use LD_LIBRARY_PATH at run time.  Because we use
> zdaemon and zdaemon lets us specify run-time environment variables,
> this isn't a problem. (Note that a Python script can't set it's own
> LD_LIBRARY_PATH, because it is too late -- the library loader has
> already looked for this before a Python script can set it.  This
> works with zdaemon because zdaemon creates a separate subprocess for
> the application and the environment variables are set before the
> subprocess runs.)

Ah; luckily we're using zdaemon as well, so I'll go down that path.  I
suppose I could combine what Deliverance does with zdaemon and have a
buildout which:

* builds libxml2 and libxslt
* builds lxml against them
* updates my zdaemon.conf to include the resulting lib path


>
> Jim
>
> --
> Jim Fulton                      mailto:jim at zope.com             Python Powered!
> CTO                             (540) 361-1714                  http://www.python.org
> Zope Corporation        http://www.zope.com             http://www.zope.org
>
>
>
>