[pypy-dev] cpyext - compilation, and swig?

[Darjus Loktevic]

After looking at SWIG cffi example, it seems that adding basic support
for PyPy ctypes should not be hard (no C++ support of course). I might
play with it on the weekend.

Darjus

On Fri, May 13, 2011 at 1:27 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> Hi,
>
> 2011/5/13 Dan Stromberg <drsalists at gmail.com>:
>>
>> When I build a swig-generated C extension module against CPython 2.6, and
>> attempt to load it using cpyext, I get:
>>
>> /usr/local/pypy-1.5/bin/pypy
>>>>>> cpyext.load_module('_odirectcmodule.so', 'odirect')
>> Traceback (most recent call last):
>>   File "<console>", line 1, in <module>
>> ImportError: unable to load extension module './_odirectcmodule.so':
>> ./_odirectcmodule.so: undefined symbol: PyClass_Type
>>>>>>
>>
>> 1) Is that the right way to load a C extension module using cpyext?
>
> No, you should just import it, like with CPython.
>
>> 2) Do I need to recompile the module for pypy?
>
> Yes, because the memory layout of objects is different, and some
> macros are turned into function calls.
> And to enforce the difference with CPython, the extension modules are
> named differently: mymodule.pypy-15.so
>
> Normally, it's enough to run "pypy setup.py install"
>
>> 3) Are there known issues with cpyext and swig?
>
> Some SWIG extension modules are known to work.
> Normally it should compile and work normally,
> except if you have custom typemaps and other C
> code that uses the CPython API.
>
>> 4) Of swig, Cython, hand-coded C, and ctypes, which is the better bet today
>> for reliability with use from PyPy?
>
> Cython won't work with pypy because it plays too much with the CPython
> internals.
>
> I don't have an opinion about it, but some of us consider ctypes as the best way
> to develop extension modules. ctypes will eventually become much
> faster with pypy,
> with the JIT compiler emitting directly the machine code to call the function.
> In any case, only a pure Python implementation can be sped up by the JIT.
> A C function cannot be traced...
>
> At the moment, cpyext is still considered as a (useful) experiment that helps
> you leverage existing C extensions, but it will always be slow for different
> reasons, the most obvious one is that the garbabe collector used by pypy
> moves objects, whereas PyObject* pointers are expected to stay in place...
>
> Also, there are probably other directions that we have not yet explored:
> - implement another C API better suited to pypy
> - a SWIG backend which emits ctypes code or similar
> And probably others...
>
> --
> Amaury Forgeot d'Arc
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>