[Distutils] python eggs and lxml

[Phillip J. Eby]

At 11:57 PM 9/23/2005 +0200, Martijn Faassen wrote:
>I've been looking at Python eggs, easy install and setuptools with a lot
>of interest -- very impressive work. I've been thinking of packaging
>lxml with it, and, on the larger scale, look into packaging Zope 3 with it.
>
>Concerning lxml I run into a few questions however.
>
>lxml depends on large external C libraries (libxml2 and libxslt).
>
>a) Is there a way to require versions of C libraries to be available in
>the Python eggs dependencies? I can't seem to find a reference to this
>scenario, but perhaps I didn't look carefully enough. The goal here
>would be to give users trying to install lxml (or something that depends
>on lxml) useful feedback about what in their system they're missing (or
>have the wrong version of).

Your options here are the same as with any distutils package, which is to 
say you have to figure it out yourself.  ;)  You can add code to look for 
the libraries, embed your own source, etc.


>b) Going further, it'd be nice for some scenarios to actually be able to
>include private versions of libxml2/libxslt in a Python eggs. This is
>especially interesting on Windows deployments (where you'd include a
>binary of these libraries). Has something like this considered? I saw
>references to Pyrex support, but in lxml's case, the Pyrex code depends
>on a large underlying library.

You can certainly do that; just list the appropriate .c files in your 
Extension.

For PEAK on Python 2.3, I include an expat wrapping that adds the Python 
2.4 pyexpat features this way, using something like:

     Extension("peak.util.pyexpat", [
             "src/peak/util/pyexpat.c", "src/expat/xmlparse.c",
             "src/expat/xmltok.c", "src/expat/xmlrole.c",
         ],
         include_dirs=["src/expat"],
         define_macros=[('XML_STATIC',1),('HAVE_MEMMOVE',1)]   # XXX
     ),


>c) It's also interesting though for deployment on linux. It'd be nice to
>be able to include the source versions of specific versions of libxml2
>and libxslt with lxml and to be able to build/install them such that
>they are only used for lxml. This way the system libraries (which may be
>out of date or have otherwise a wrong version) would not be in play and
>wouldn't be affected.

Yeah, just bake it in as shown above.


>If something like this were arranged, it'd be much easier to make lxml a
>requirement of a large package like for instance Zope 3 (which is being
>considered).
>
>I realize that any or all of these might be out of scope for easy
>install -- in the Linux case, it might be deferred to a Linux package
>management system, for instance. Still, I imagine the case where a
>Python library has a dependency on a potentially large non-Python
>codebase could be fairly common, and it'd be nice if such libraries
>could be "first class" easy_install citizens so that other Python
>libraries can safely depend on them. What are people's thoughts were
>about supporting such scenarios?

Not all libraries can be bundled by source, of course.  Sometimes you 
really need to use whatever the "system version" is, for one reason or 
another.  Database clients, for example, are something you really really 
want to use the local version for.  I'm thinking that the distutils could 
really use some sort of library-finding capabilities for that stuff, 
assuming they don't already have some I just haven't found yet.