Mailman 3 May 2011 - cython-devel

Re: [Cython] adding support for __dict__ in extension types
by Lisandro Dalcin Sept. 26, 2011

Sept. 26, 2011

Bringing up this old post... On 21 June 2010 15:41, Robert Bradshaw <robertwb(a)math.washington.edu> wrote: > On Jun 17, 2010, at 9:31 AM, Lisandro Dalcin wrote: > >> If we special case a __dict__ attribute in extension types, i.e: >> >> cdef class Foo: >> cdef dict __dict__ >> >> and fill type->tp_dictoffset, then we can support __dict__ in >> extension types. >> >> What do you think? > > Sounds like a good idea to me. Note that we check tp_dictoffset for > fast dispatching for cpdef methods (which would be correct as a dict > lookup *would* be needed if __dict__ is available). > I still have this patch lying around in my disk. I remember Stefan had some objections. For example, when the user ask for __dict__, a new dict is unconditionally created (in CPython, type dict are allocated on-demand). I propose to get this patch pushed now, and optimize later (however, I really don't know how to safely implement this optimization). -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo 3000 Santa Fe, Argentina Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169

3 4

Bindings performance issue
by Vitja Makarov June 21, 2011

June 21, 2011

Hi! Is bindings performance issue valuable? $ cat bindbench.pyx def wo_bindings(): pass def outer(): def inner(): pass return inner with_bindings = outer() $ python >>> import timeit >>> timeit.repeat('with_bindings()', setup='from bindbench import wo_bindings, with_bindings', repeat=1, number=100000000) [6.169871807098389] >>> timeit.repeat('wo_bindings()', setup='from bindbench import wo_bindings, with_bindings', repeat=1, number=100000000) [4.609416961669922] PyCBindings makes it 1.3 (difference is about 15ns on my laptop) times slower for CPython interpreter execution. As CPython has some optimizations for CFunctions and PyCFunctions. Does it make sense for us? Or we can easily switch to bindings? -- vitja.

4 26

local variable handling in generators
by Stefan Behnel June 12, 2011

June 12, 2011

Hi, I've been looking at the nqueens benchmark for a while, and I think it's actually not that a bad benchmark for generators. http://hg.python.org/benchmarks/file/tip/performance/bm_nqueens.py A better implementation only for Py2.7/Py3 is here: https://github.com/cython/cython/blob/master/Demos/benchmarks/nqueens.py Cython currently runs the first implementation about as fast as Py3.3: https://sage.math.washington.edu:8091/hudson/job/cython-devel-pybenchmarks-… and the second one more than 3x as fast: https://sage.math.washington.edu:8091/hudson/view/bench/job/cython-devel-cy… However, I think there's still some space for improvements, and local variables are part of that. For generator functions that do non-trivial things between yields, I think that local variables will quickly become a bottleneck. Currently, they are always closure fields, so any access to them will use a pointer indirection to a foreign struct, originally passed in as an argument to the function. Given that generators often do Python object manipulation through C-API calls, any such call will basically require the C compiler to assume that all values in the closure may have changed, thus disabling any optimisations for them. The same applies to many other object related operations or pointer operations (even DECREF!), as the C compiler cannot know that the generator function owns the closure during its lifetime exclusively. I think it would be worth changing the current implementation to use local C variables for local Cython variables in the generator, and to copy the values back into/from the closure around yields. I'd even let local Python references start off as NULL when the generator is created, given that Vitek's branch can eliminate None initialisations now. I started looking into this a bit, but it's not a quick change. The main problem I see is the current code duplication between the generator body node and the DefNode. It would be better if both could share more code. Basically, if local variables become truly local, the only differences will be that they come from call arguments in one case and from the closure in the other, and that generators have the additional jump-to-yield entry step. Everything else could hopefully be identical. Once again, this goes hand in hand with the still pending DefNode refactoring... Stefan

4 20

[GSoC] Blog post regarding the Cython backend aiming PyPy
by Romain Guillebert May 31, 2011

May 31, 2011

Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.h… Cheers Romain

2 2

wiki.cython.org is down
by Vitja Makarov May 31, 2011

May 31, 2011

With error: Service Temporarily Unavailable The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. -- vitja.

2 1

[GSoC] Blog post regarding the Cython backend aiming PyPy
by Romain Guillebert May 30, 2011

May 30, 2011

Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.h… Cheers Romain

1 0

[GSoC] Blog post regarding the Cython backend aiming PyPy
by Romain Guillebert May 30, 2011

May 30, 2011

Hi I've posted and article on my blog that explains what I've done during the community bonding period and the first week of the Google Summer of Code : http://rguillebert.blogspot.com/2011/05/cython-backend-aiming-pypy-week-1.h… Cheers Romain

1 0

Control Flow
by Vitja Makarov May 30, 2011

May 30, 2011

I've recently fixed some issues: - closure variables were not tracked - scoped expression variables were initialized to None So, I should fix broken tests, here are some of them: nogil (should be fixed) purecdef (upstream) cfunc_directive_in_pyclass (upstream) bufaccess_noassignT444 (should be removed?) bufaccess (should be fixed) ass2longlong (should be fixed) builtinfuncs (should be fixed) extcmethcall (should be fixed) extcoerce (fixed or removed?) e_extweakref (should be fixed) -- vitja.

4 22

Redundant Cython exception message strings
by Stefan Behnel May 28, 2011

May 28, 2011

Hi, I recently stumbled over a tradeoff question with AttributeError, and now found the same situation for UnboundLocalError in Vitja's control flow branch. So here it is. When we raise an exception several times in different parts of the code with a message that only differs slightly each time (usually something like "'NoneType' has no attribute X", or "local variable X referenced before assignment"), we have three choices to handle this: 1) Optimise for speed: create a Python string object at module initialisation time and call PyErr_SetObject(exc_type, msg_str_obj). 2) Current way: let CPython create the string object when raising the exception and just call PyErr_SetString(exc_type, "complete message"). 3) Trade speed for size and allow the C compiler to reduce the storage redundancy: write only the message template and the names as C char* constants by calling PyErr_Format(exc_type, "message template %s", "X"). Assuming that exceptions should be exceptional, I'm leaning towards 3). This would allow the C compiler to collapse multiple usages of the same C string into one data constant, thus reducing a bit of redundancy in the shared library size and the memory footprint. However, it would (slightly?) slow down the exception raising due to the additional string formatting, even when compared to the need to build a Python string object that it shares with 2). While 1) would obviously be the fastest way to raise an exception (no memory allocation, only refcounting), I think it's not worth it for exceptions as it increases both the runtime memory overhead and the module startup time. Thoughts? Stefan

3 6

Broken tests
by Stefan Behnel May 27, 2011

May 27, 2011

Hi, could we please make it a requirement to run the complete test suite after a branch merge? Or at least taking a close look at Jenkins after pushing a merge? And then fixing the bugs it finds? Stefan

4 4