[Cython] Conditional import in pure Python mode

Robert Bradshaw robertwb at gmail.com
Wed May 2 09:59:46 CEST 2012


On Wed, May 2, 2012 at 12:33 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 02.05.2012 08:56:
>> On Tue, May 1, 2012 at 1:02 PM, Stefan Behnel wrote:
>>> Francesc Alted, 01.05.2012 21:49:
>>>> On 5/1/12 2:39 PM, mark florisson wrote:
>>>>> On 1 May 2012 20:22, Stefan Behnel wrote:
>>>>>> Stefan Behnel, 01.05.2012 21:14:
>>>>>>> 2) Use math.pxd as an override for the math module. I'm not sure yet how
>>>>>>> that would best be made to work, but it shouldn't be all that complex. It
>>>>>>> already works (mostly?) for numpy.pxd, for example, although that's done
>>>>>>> explicitly in user code.
>>
>> math.pxd would be a bit trickier, as we're trying to shadow python
>> functions with independent c implementations (rather than declaring
>> structure to the single numpy array object and exposing c-level only
>> methods. We'd need to support stuff like
>>
>> double x = ...
>> double y = sin(x) # fast
>> cdef object f = sin # grab the builtin one?
>>
>> but this is by no means insurmountable and could be really useful.
>
> I already did that for the builtin abs() function. Works nicely so far,
> although not from a .pxd but declared internally in Builtin.py.
>
> It's not currently supported for methods (I tried it for one of the builtin
> types and it seemed to require more work than I wanted to invest at that
> point), but I don't think we need that here. Module level functions should
> totally be enough for math.pxd.

Yep.

>>>>>> BTW, I think it would be helpful to make the numpy.pxd cimport automatic as
>>>>>> well whenever someone does "import numpy" and friends, right?
>>>>>>
>>>>> I'm not sure, it means the user has to have numpy development headers.
>>>>
>>>> But if the user is going to compile a NumPy application, it sounds like
>>>> strange to me that she should not be required to install the NumPy
>>>> development headers, right?
>>>
>>> Let's say it's not impossible that someone uses NumPy and Cython without
>>> any accelerated C level connection between the two, but it's rather
>>> unlikely, given that Cython already has all dependencies that this
>>> connection would require as well, except for the NumPy header files.
>>>
>>> So I would suggest to make the automatic override the default for any
>>> module for which a .pxd file with the same fully qualified name is found in
>>> the search path, and to require users to explicitly disable this feature
>>> for a given module using a module level (or external) compiler directive if
>>> they feel like getting slower code (or working around a bug or whatever).
>>
>> There is another consideration: this can introduce unnecessary and
>> potentially costly dependencies. For example, in Sage one has
>> sage/rings/integer.pxd. Not everything that imports from this file
>> needs c-level access to the Integer type, and requiring everything
>> that imports from sage.rings.integer to be re-compiled when this file
>> changes would increase the (admittedly already lengthy) re-compile, as
>> well as sucking in a (chain of) un-needed declarations.
>
> Ah, yes, I see your point. The .pxd may actually introduce substantially
> more (and different) dependencies than the already compiled .so file. Just
> because the import happens in Cython code and not in Python code doesn't
> mean it should do different things.
>
>
>> As Cython
>> becomes more and more common, a similar effect could happen between
>> projects as well.
>
> Agreed. A compile time C level dependency is much more fragile and version
> dependent than a Python level import. I can see a lot of cases where this
> matters.
>
>
>> This may be the exception rather than the rule, so perhaps it's not
>> *that* bad to let opt-in be the default, i.e.
>>
>> # cython: cimport_on_import = __all__
>
> So, you would allow it to receive either a sequence of explicit module
> names or "__all__" to enable it by default, right? Sounds like a reasonable
> directive to me.

Perhaps the default could be "those that ship with Cython" or even
some other hand-picked list. (In this case, we'd want users to be able
to add to and remove from the default set, e.g.

# cython: cimport_on_import = +my_module, -math

It'd also be nice to be able to specify a package and all its submodules...)

- Robert


More information about the cython-devel mailing list