[Cython] PyCon-DE wrap-up by Kay Hayen

[mark florisson]

On 10 October 2011 09:38, Stefan Behnel <stefan_ml at behnel.de> wrote:
> mark florisson, 09.10.2011 19:57:
>>
>> On 9 October 2011 18:35, Stefan Behnel wrote:
>>>
>>> One of the impressions I took out of the technical discussions with Kay
>>> is
>>> that there isn't really a good reason why Cython should refuse to
>>> duplicate
>>> some of the inner mechanics of CPython for optimisation purposes. Nuitka
>>> appears to be somewhat more aggressive here, partly because Kay doesn't
>>> currently care all that much about portability (e.g. to Python 3).
>>
>> Interesting. What kind of (significant) optimizations could be made by
>> duplicating code? Do you want to duplicate entire functions or do you
>> want to inline parts of those?
>
> I was mainly referring to things like direct access to type/object struct
> fields and little things like that.

Ah, I see. I suppose that if you do everything through Cython-specific
macros it will be easy to change it at any time and it will make it
easy to experiment with performance as well.

> They can make a difference especially in
> loops, compared to calling into a generic C-API function. For example, we
> could have our own interned implementation of PyDict_Next(). I'm not very
> impressed by the performance of that C-API function - repeated calls to
> GetItem can be faster than looping over a dict with PyDict_Next()!
>
> That being said, I wasn't referring to any specific changes. It was more of
> a general remark about the invisible line that we currently draw in Cython.
>
>
>> I actually think we should not get too tied to CPython, e.g. what if
>> PyPy gets a CPython compatible API
>
> It already implements a part of the C-API:
>
> http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.html
>
> However, if we really want to support it at that level, there's likely more
> to do than just removing low-level optimisations. And that would take the
> normal route that we always use: macros and conditionally compiled inline
> functions. The mere fact that we try to support different targets doesn't
> mean that we should stop optimising for specific targets. The same is true
> for different versions of CPython, where we often use better optimisations
> in newer releases, without sacrificing backwards compatibility.
>
> Personally, I think that supporting PyPy at the Python level is a lot more
> interesting, although it may be easier to get it working at the cpyext
> level.
>

Yeah it's certainly interesting. It might be hard to support things
like cython.parallel and efficient buffer access though. I think
releasing the GIL might not be very easy either, although perhaps that
could be circumvented by factoring the entire nogil block out into a C
function which you call with ctypes.

>> or possibly a subset like PEP 384?
>
> That's currently not very interesting since there are basically no C
> extensions around (generated or hand written) that restrict themselves to
> that API. Supporting it in Cython would mean that we have to rewrite huge
> parts of the generated C code. It's not even clear to me yet that we *can*
> implement all of Cython's features based on PEP 384. For example, fast
> indexing into lists and tuples is basically a no-no in the restricted C-API.
> There are tons of rather unexpected restrictions like this.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>