[Distutils] c extensions: how to rebuild and test quickly

Paul Pogonyshev pogonyshev at gmx.net
Thu Feb 15 22:01:57 CET 2007


Thomas Heller wrote:
> Paul Pogonyshev schrieb:
> > Thomas Heller wrote:
> >> Paul Pogonyshev schrieb:
> >> > Hi,
> >> > 
> >> > Distutils create dynamic library for a C extension in `build/...'
> >> > directory.  This makes it impossible to run program without
> >> > installing it, which I find very important for developing.  Ideally,
> >> > process should look like this:
> >> > [...]
> >>
> >> What I do is to create a loader module named <extension>.py that  
> >> 'replaces' itself with the real extension, loading it via imp
> >> from the platform specific build directory that distutils creates.
> > 
> > This looks clever.  However, how do you make sure the module doesn't
> > get itself installed?  I.e. I'd not want this hack make it into the
> > installed copy, while it is perfectly OK for quick in-place builds
> > and tests.
> 
> I'm not sure this trick works for every case, but it works for the ctypes
> separate distribution.
> 
> The point is that ctypes is a package, and the ctypes package uses the
> _ctypes.pyd/so and _ctypes_test.pyd/.so extension modules.  The extension
> modules are not inside the package, they are top-level modules.  Here
> is the source layout in the SVN repository:
> 
> ./setup.py
> ./_ctypes.py            # the loader module
> ./_ctypes_test.py       # the other loader module
> ./ctypes/__init__.py    # the ctypes package
> ./ctypes/...            # other parts of the package
> 
> Now, when you call 'setup.py build_ext', the extension modules
> are built and put into this directory:
> 
> ./build/lib.win32-2.4/_ctypes.pyd
> ./build/lib.win32-2.4/_ctypes_test.pyd
> 
> The setup script contains these lines:
> 
> setup(...,
>       extensions = [Extension(_ctypes, ...), Extension(_ctypes_test, ...)],
>       packages = ["ctypes", "ctypes.test", "ctypes.macholib"],
>       )
> 
> so the loader script will not be included in the distributions.
> 
> I am not sure if this recipe can be extended to support extensions
> that are inside packages; but for now I find it nicer than the setuptools
> 'develop' approach.
> 
> Thomas

Hm, looks too complicated.  Thanks anyway.

I ended up with custom build_ext command:

from distutils.command.build_ext import build_ext as _build_ext

class build_ext (_build_ext):

    def build_extension (self, extension):
        _build_ext.build_extension (self, extension)

        if not self.inplace and os.name == 'posix':
            filename = self.get_ext_filename (extension.name)

            try:
                os.remove (filename)
            except:
                # Ignore all errors.
                pass

            try:
                os.symlink (os.path.join (os.pardir, self.build_lib, filename),
                            os.path.join ('notify', os.path.basename (filename)))
            except:
                # Ignore all errors.
                pass

Maybe I should copy on non-posix systems.

Paul


More information about the Distutils-SIG mailing list