[Cython] Utility Codes and templates
stefan_ml at behnel.de
Tue Jul 26 08:50:09 CEST 2011
Robert Bradshaw, 26.07.2011 07:00:
> On Mon, Jul 25, 2011 at 9:48 PM, Vitja Makarov wrote:
>> I think that would seriously help with moving shared C-code into
>> cython library (.h and .so).
>> Some things like generator class implementation and cyfunction could
>> be move into .so file.
> Yes, that would be good.
Well, *if* we find a way to make sure the additional modules get properly
distributed. It's not obvious to me how to do that, certainly not in a
cross-package way, and not even in a per-package way, as that would still
require some kind of interaction with distutils and maybe even changes to
the users' setup.py scripts (hopefully in a way that doesn't require them
to know what exact additional modules will be generated).
> Of course we don't want a single .so file for
> every tiny chunk of utility code, but one needs to be careful lumping
> them together as well (e.g. things may compile differently depending
> on the set of headers included in the .pyx file). To do this right I
> we need to compile things at a higher level than module-by-module with
> Cython.Build.cythonize or similar.
> Certainly CyFunction and the generator class are easy and high-benefit
> targets (compared to, e.g. tuple indexing utility code).
Now that we have a way to group utility code in a single file, including
automatically resolved dependencies to other files, it should be trivial to
take such a file and to compile it into a separate module. Cython modules
would then trivially export their public names anyway, and utility code in
C files could have an additional Cython wrapper file that would simply
depend on the C file and that properly exports the C functions through a
Python API and inter-module C-API.
We could even write empty utility files that only contain dependencies,
thus grouping together multiple utility files to one larger module.
More information about the cython-devel