[XML-SIG] Disposition of C extensions and packages

[Thomas B. Passin]

A.M. Kuchling wrote:

> [Crossposted to xml-sig, distutils-sig]
>
> I'm working on getting the XML-SIG's CVS tree to install using the
> current version of the Distutils.  Right now there are two C
> extensions, sgmlop.so and pyexpat.so, and they're installed under
> xml/parsers/ .  It's hard to handle this case using the distutils code
> as it stands, because it expects to put extensions into a
> build/platlib/ directory, from which they'll be installed into
> site-packages.
>
> I can coerce setup.py into installing them into xml/parsers/, by
> subclassing the BuildExt command and setting build_dir myself:
>
>     from distutils.command.build_ext import BuildExt
>     class XMLBuildExt(BuildExt):
>         def set_default_options (self):
>             BuildExt.set_default_options( self )
>             self.build_dir = 'lib/build/xml/parser'
>
>     setup (name = "PyXML", cmdclass = {'build_ext':XMLBuildExt}, ...)
>
> You also have to subclass the Install command and set build_dir
> there; I've trimmed that code.  It's really clunky.\
>
> Note that this scheme will break once there are C modules that need to
> be installed anywhere other than xml/parsers/, because build_dir is
> being hardwired without knowledge of what module is being compiled.
>
> Questions:
>
> 1) A general Python question about packaging style: Is mixing
>            C extensions and Python modules in one package tree a bad
>            idea?  It makes the whole tree platform-dependent, which is
>            probably annoying for sites maintaining Python installation
>            for different architectures.
>
> 2) Another general question, this time o: how should this be
>            handled?  Should C extensions always be effectively
>            top-level, and therefore go into site-packages?  Should
>            there be an xml package holding .py files, and an X package
>            holding all the C extensions? (X = 'plat_xml',
>            'xml_binary', or something like that)
>
Don't forget Windows! Windows users need a working package with binaries
that installs easily.  In Windows there is a common directory for DLLs, and
no path changes are needed if the binaries go there.  for unix/linux, why
not put binaries wherever the python .so files goes (I'm assuming there is
one, but I seem to remember seeing one on a linux installation)?  How are
others planning to handle this (other SIGs and future Python releases)?
It's going to keep coming up, why not have a common solution?  Of course,
this distribution might need to come out before there is an agreement....,
I do agree that the binaries shouldn't be mixed into the .py files in the
tree.


> 3) XML-SIG question: should I go ahead and change it (since I
>            first changed it to use xml.parsers.sgmlop)?
>
> 4) Distutils question: is this a problem with the Distutils
>            code that needs fixing?  I suspect not; if the tools make
>            it difficult to do stupid things like mix .py and .so
>            files, that's a good thing.
>
> --
Let's thank A. M. for doing all this work!

Tom Passin