[Distutils] c extensions: how to rebuild and test quickly

[Thomas Heller]

Paul Pogonyshev schrieb:
> Thomas Heller wrote:
>> Paul Pogonyshev schrieb:
>> > Hi,
>> > 
>> > Distutils create dynamic library for a C extension in `build/...'
>> > directory.  This makes it impossible to run program without
>> > installing it, which I find very important for developing.  Ideally,
>> > process should look like this:
>> > [...]
>>
>> What I do is to create a loader module named <extension>.py that  
>> 'replaces' itself with the real extension, loading it via imp
>> from the platform specific build directory that distutils creates.
> 
> This looks clever.  However, how do you make sure the module doesn't
> get itself installed?  I.e. I'd not want this hack make it into the
> installed copy, while it is perfectly OK for quick in-place builds
> and tests.

I'm not sure this trick works for every case, but it works for the ctypes
separate distribution.

The point is that ctypes is a package, and the ctypes package uses the
_ctypes.pyd/so and _ctypes_test.pyd/.so extension modules.  The extension
modules are not inside the package, they are top-level modules.  Here
is the source layout in the SVN repository:

./setup.py
./_ctypes.py            # the loader module
./_ctypes_test.py       # the other loader module
./ctypes/__init__.py    # the ctypes package
./ctypes/...            # other parts of the package

Now, when you call 'setup.py build_ext', the extension modules
are built and put into this directory:

./build/lib.win32-2.4/_ctypes.pyd
./build/lib.win32-2.4/_ctypes_test.pyd

The setup script contains these lines:

setup(...,
      extensions = [Extension(_ctypes, ...), Extension(_ctypes_test, ...)],
      packages = ["ctypes", "ctypes.test", "ctypes.macholib"],
      )

so the loader script will not be included in the distributions.

I am not sure if this recipe can be extended to support extensions
that are inside packages; but for now I find it nicer than the setuptools
'develop' approach.

Thomas