[Distutils] Disposition of C extensions and packages

[M.-A. Lemburg]

Michel Sanner wrote:
> 
> On Dec 20, 12:06am, M.-A. Lemburg wrote:
> > Subject: Re: [Distutils] Disposition of C extensions and packages
> > "A.M. Kuchling" wrote:
> > >
> > > Questions:
> > >
> > >         1) A general Python question about packaging style: Is mixing
> > >            C extensions and Python modules in one package tree a bad
> > >            idea?  It makes the whole tree platform-dependent, which is
> > >            probably annoying for sites maintaining Python installation
> > >            for different architectures.
> >
> > I have been using that setup for two years now with all of
> > my mx extensions and so far it has been working great.
> >
> > If you maintain packages with C extensions for several
> > platforms, you can simply install the packages under
> > the platform subdirs in /usr/local/lib/python1.5 -- one
> > copy for every platform. Disk space is no argument anymore
> > nowadays.
> >
> 
> Agreed, disk space is not an issue, but file duplication is not very nice. I
> think this is a real weakness in Python: on one hand we have platform
> independent extensions which are great on the other hand we have native
> extensions when performance is required. This is, at least for me (in
> scientific computing) what makes Python such a great tool, BUT Python does not
> provide a mechanism to split a single package into platform dependent and
> platform independent part. We maintain Python and a number of packages for
> IRIX, SunOS, Dec Alpha OSF, Win32 and linux ... and having to update 5 .py
> files at every bug fix is a real pain.

One way to solve this is by editing the __init__.py module
of the package containing the C extension and tweaking the
__path__ global so that the correct shared modules for the
importing platform is found. I've never tried this, but it
should work...

Note that at least Linux .so files and Windows .pyd files
can live happily side-by-side in one directory.

With distutils in place this issue should basically disappear
altoghether, since then the whole building process would
be automated -- not sure whether distutils allows having
different setups for different architectures, but given that
it is written in Python, it should be possible to automate
all aspects of multi-platform installations.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                    11 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/