[Cython] Multiple modules in one compilation unit

Lisandro Dalcin dalcinl at gmail.com
Wed Mar 2 19:58:43 CET 2011

On 2 March 2011 13:01, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Dag Sverre Seljebotn, 02.03.2011 16:37:
>> On 03/02/2011 04:11 PM, Lisandro Dalcin wrote:
>>> On 2 March 2011 08:35, Stefan Behnel<stefan_ml at behnel.de> wrote:
>>>> Dag Sverre Seljebotn, 02.03.2011 11:54:
>>>>> Problem is that Fortran code often has...interesting...programming
>>>>> practices. Global variables abound, and are often initialised between
>>>>> modules. Imagine:
>>>>> settings_mod.set_alpha(0.34)
>>>>> print compute_mod.get_alpha_squared()
>>>>> This behaves quite differently with two static versions rather than
>>>>> one...
>>>> Then I'd suggest always linking dynamically.
>>> And where are you going to put your fortran shared libraries? Dynamic
>>> linking details varies wildly across platforms... I'm very much
>>> understand Dag's use case and concerns, and I do think that some
>>> research in all this is worth it.
>> I'm not sure if there's much more to research at the moment -- Stefan says
>> it is possible, and that's what I wanted to know at this stage. If I want
>> it, I obviously need to implement it myself. (And if such a patch
>> implements PEP 3121 and there's a demonstrated need for it with some
>> users,
>> I really can't see it getting rejected just out of it being in "poor
>> taste").
>> I.e., I'm going to make Fwrap spit out multiple pyx files and worry about
>> this later. If multiple .pyx in one .so was fundamentally impossible, I
>> might have gone another route with Fwrap. That was all.
> The feature I could imagine becoming part of Cython is "compiling packages".
> I.e. you'd call "cython" on a package and it would output a directory with a
> single __init__.so that contains the modules compiled from all .pyx/.py
> files in that package. Importing the package would then trigger an import of
> that __init__.so, which in turn will execute code in its init__init__()
> function to register the other modules.

$ cd /tmp/
$ mkdir pkg
$ touch pkg/__init__.so
$ python -c 'import pkg'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named pkg

So that's not going to work... The import machinery ignores
__init__.so files. Instead, you need a "package.so" file, the
initpackage() init function should in turn setup the package structure
and initialize submodules. I've sent privately to Dag an example of
how this should be done.

Lisandro Dalcin
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

More information about the cython-devel mailing list