[Cython] compiler performance issue for extended utility code

Vitja Makarov vitja.makarov at gmail.com
Sat Oct 8 14:10:53 CEST 2011

2011/10/8 mark florisson <markflorisson88 at gmail.com>:
> On 8 October 2011 08:03, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Vitja Makarov, 07.10.2011 18:01:
>>>> 2011/10/7 Stefan Behnel:
>>>>> Vitja Makarov, 06.10.2011 23:12:
>>>>>> Here is small comparison on compiling urllib.py with cython:
>>>>>> ((e8527c5...)) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>>>>>> ../cython.py urllib.py
>>>>>> real    0m1.699s
>>>>>> user    0m1.650s
>>>>>> sys     0m0.040s
>>>>>> (master) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>>>>>> ../cython.py urllib.py
>>>>>> real    0m2.830s
>>>>>> user    0m2.790s
>>>>>> sys     0m0.030s
>>>>>> It's about 1.5 times slower.
>>>>> That's a pretty serious regression for
>>>>> plain Python code then. Again, this needs proper profiling.
>>> I've added return statement on top of CythonScope.test_cythonscope,
>>> now I have these timings:
>>> (master) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>>> ../cython.py urllib.py
>>> real    0m1.764s
>>> user    0m1.700s
>>> sys     0m0.060s
>> Ok, then it's only a bug. "create_testscope" is on by default in Main.py,
>> Context.__init__(). I don't know what it does exactly, but my guess is that
>> the option should a) be off by default and b) should rather be passed in by
>> the test runner as part of the compile options rather than being a parameter
>> of the Context class. AFAICT, it's currently only used in TreeFragment.py,
>> where it is being switched off explicitly for parsing code snippets.
>> Stefan
>> _______________________________________________
>> cython-devel mailing list
>> cython-devel at python.org
>> http://mail.python.org/mailman/listinfo/cython-devel
> It turns it off to avoid infinite recursion. This basically means that
> you cannot use stuf from the Cython scope in your Cython utilities. So
> in your Cython utilities, you have to declare the C version of it
> (which you declared with the @cname decorator).
> This is not really something that can just be avoided loading like
> this. Perhaps one solution could be to load the test scope when you do
> a lookup in the cython scope for which no entry is found. But really,
> libcython and serializing entries will solve all this, so I suppose
> the real question is, do we want to do a release before we support
> such functionality?
> Anyway, the cython scope lookup would be a simple hack worth a try.

Does utility code supports something like dependencies? And could that
help here?

I've also noticed that some utilities are loaded unconditionally
perhaps it's better to introduce lazy loading.


More information about the cython-devel mailing list