[Numpy-discussion] Exporting Numpy C functionality to other extension modules

[Todd Miller]

Chris Barker wrote:

>Jack Jansen wrote:
>
>  
>
>>An extension module that knows about this protocol and gets passed an
>>object that it think might be a Numeric array checks whether the object
>>has an __Numeric_C_interface attribute. If so it retrieves it, checks
>>that it is a Cobject, gets the descriptor and tests it for
>>compatibility and if it is compatible gets the cobject pointer and
>>happily calls all the Numeric routines it needs.
>>    
>>
>
>Wow Jack! are single handely going to impliment all my pet projects that
>I'm too stupid to know how to do my self ? (the other one was Universal
>text file support)
>
>I can only barely follow what you're suggesting, but I still have a
>question about it. It seems while this would provide a way ro an
>extension module to identify whether an object was a Numeric array, and
>then get a pointer to it, how would it know the API for dealing with the
>arrays, without the Numeric header file? Or would you have to include
>the header file when compiling, but not need the library at runtime
>unless it was actually used, which seems a reasonable compromise.
>
>If this would work, I think it's a great idea. Short of including
>NumArray with the standard library (which I imagine is a least a couple
>of Python releases away), it would be a great solution for folks that
>are writing extensions that they want to be able take advantage of
>Numeric when it's there, but not require it.
>
>Do any of the primary Numarray developers think this is a good and
>doable idea?
>  
>
Roll out the time machine...  it's already done.

As long as you don't define the macros PY_ARRAY_UNIQUE_SYMBOL or 
NO_IMPORT_ARRAY,  any file that includes arrayobject.h gets a static 
copy of PyArray_API.

If the module executes import_array() at an appropriate time,  normally 
module initialization, but not necessarily,  the static PyArray_API gets 
filled in and becomes usable.    The import_array() call is critical; 
 without it,  API calls through the static PyArray_API are calls to NULL 
and segfault.

I think that if Numeric is not present,  and you call import_array(),   
it will fail quietly but leave the Python error status set.   So it 
might make sense to call PyErr_Clear() after doing import_array().  

>-Chris
>
So it sounds like your whole "weak linkage" scheme is plausible now with 
Numeric (maybe even numarray!), as would be a minimal API module.

1.  We discussed yesterday how to determine if an object is a Numeric 
array w/o even compiling with arrayobject.h.   The important idea there 
was that if Numeric is not present,  the "isarray" (or whatever) 
function will return false rather than segfaulting because the API 
pointer isn't filled in.

2. Call API functions in contexts where you know you're looking at 
Numeric arrays, i.e.,  right after isarray().  This creates a guard 
which prevents you from calling API functions when Numeric is not present.

3.  Call import_array() at some time before using the API functions, 
 possibly at module init time, failing quietly and clearing the error in 
installations where Numeric is not installed.


Todd