[Cython] Generators & closure optimization
stefan_ml at behnel.de
Wed Dec 21 21:17:23 CET 2011
Vitja Makarov, 21.12.2011 19:48:
> Some time ago we were talking about generators optimization by copying
> local variables from closure into local scope.
Yes, I think that will make it easier for the C compiler to make optimistic
assumptions about external values.
> Now I think that should be a good idea to implement this for both
> generators and regular closure functions. So local var will be used
> for reference and assignment should be made to both copies. Of course
> there are some variables that shouldn't be copied: non-local vars,
> arrays, C++ classes and structures.
Basically, anything that external code can modify. That makes it a bit
tricky to do it also for 'normal' closure functions - the whole idea is
that there is more than one function that can refer to a variable.
> Also it may be a good idea to move outer scope pointer into local variable.
> So I'm wondering what is a good test to measure actual speedup?
Just take the plain Python versions of the iterparse functions and compare
them before and after the change. The raw C implementation in CPython gives
a good baseline.
Actually, it would be generally interesting to run the Cython versions
through callgrind to see where the time is actually being spent.
More information about the cython-devel