[Cython] Shared Cython runtime

Nikita Nemkin nikita at nemkin.ru
Wed Apr 10 06:22:29 CEST 2013


Robert Bradshaw, 09.04.2013 20:16:
>>> Yep. We could even create stub .so files for the included ones that
>>> did nothing but import the big one to avoid having to worry about
>>> importing things in the right order.

The stubs don't have to be shared libraries, pure python would work
just fine (and reduce the complexity).

>>> The big question is what API to specify that things should be linked
>>> together. An option to cythonize(...) could be used, but for some
>>> projects (e.g. Sage) one would want more control over which modules
>>> get bundled together rather than all-or-nothing.
>>
>> No-one forces you to use a plain cythonize("**/*.pyx"). Just be more
>> specific about what files to include in each pattern that you pass to  
>> in.
>> And you can always call cythonize() more than once if you need to. Once  
>> for
>> each meta-module should usually be acceptable, given that the bundled
>> source modules would tend to have a common configuration anyway.
>
> That would still be painful. Ideally, users should never have to
> modify the setup.py file.

Users have to plan for what to bundle, what the package structure
will be, etc. It is not an "enable and forget" type of thing.
(Unless you have an all-Cython package tree.)

I prefer explicitly creating "bundle" extensions with
distutil.core.Extension, passing multiple pyx files as sources,
then passing the result to cythonize.

If you really want to push this into cythonize (it already does more
than it should IMO), one option is to add a new comment directive,
for example:
# cython: bundle = mymodule._speedups
similar to how distutils options are passed.
All .pyx files with the same "bundle" value will be put in that bundle.

>>>> There may be glitches with stuff like "__file__" and friends, but that
>>>> should not be any worse than the current way of workings.
>>>
>>> This can probably be manipulated by Cython, though it's unclear what
>>> the best value would be.

The best value is the shared library pathname. All extension modules
have it like that. The fact that multiple modules share the same .so
is (luckily) irrelevant to the Python import system.


Best regards,
Nikita Nemkin


More information about the cython-devel mailing list