[Numpy-discussion] incorporating C/C++ code

[Tim Hochberg]

Christopher Barker wrote:
> Louis Cordier wrote:
>   
>>> At this point I would not use SWIG or Instant.
>>>       
>
> In general, SWIG makes sense if you have a substantial existing library 
> that you need access to, and particularly if that library is evolving 
> and needs to be used directly from C/C++ code as well.
>
> If you are writing C/C++ code specifically to be used as a python 
> extension, pyrex and boost::python are good choices. There was a Numeric 
> add-on to boost::python at one point, I don't know if anyone has 
> modified it for numpy.
>
>   
>> I was wondering if there where any issues with say using Psyco
>> with NumPy ?  http://psyco.sourceforge.net/
>>     
>
> Psyco knows nothing of numpy arrays, and thus can only access them as 
> generic Python objects -- so it won't help.
>
> A couple years ago, someone wrote a micro-Numeric package that used 
> python arrays as the base storage, and ran it with psyco with pretty 
> impressive results.
That might have been me. At least I have done this at least once. I even 
still have the code lying around if anyone wants to play with it. No 
guarantee that it hasn't succumbed to bit rot though.

>  What that tells me is that if psyco could be taught 
> to understand numpy arrays, (or at least the generic array interface) it 
> could work well. It would be a lot of work, however.
>   
There's another problem as well. Psyco only really knows about 2 things. 
Integers (C longs actually) and python objects (pointers).  Well, I 
guess that it also knows about arrays of integers/objects as well. It 
does not know how to handle floating point numbers directly. In fact, 
the way it handles floating point numbers is to break them into two 
32-bit chunks and store them as two integers. When one needs to operate 
on the float these two integers need to be retrieved, reassembled, 
operated on and then stuck back into two integers again. As a result, 
psyco is never going to be super fast for floating point, even if it 
learned about numeric arrays. In principle, it could learn about floats, 
but it would require a major rejiggering. As I understand it, Armin has 
no plans to do much more with Psyco other than bug fixes, instead 
working on PyPy. However, Psyco technology will likely go into PyPy 
(which I've mostly lost track of), so it's possible that down the road 
fast numeric stuff could be doable in PyPy.


-tim