[Cython] compiler performance issue for extended utility code

[mark florisson]

On 8 October 2011 08:03, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Vitja Makarov, 07.10.2011 18:01:
>>>
>>> 2011/10/7 Stefan Behnel:
>>>>
>>>> Vitja Makarov, 06.10.2011 23:12:
>>>>>
>>>>> Here is small comparison on compiling urllib.py with cython:
>>>>>
>>>>> ((e8527c5...)) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>>>>> ../cython.py urllib.py
>>>>>
>>>>> real    0m1.699s
>>>>> user    0m1.650s
>>>>> sys     0m0.040s
>>>>> (master) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>>>>> ../cython.py urllib.py
>>>>>
>>>>> real    0m2.830s
>>>>> user    0m2.790s
>>>>> sys     0m0.030s
>>>>>
>>>>>
>>>>> It's about 1.5 times slower.
>>>>
>>>> That's a pretty serious regression for
>>>> plain Python code then. Again, this needs proper profiling.
>>
>> I've added return statement on top of CythonScope.test_cythonscope,
>> now I have these timings:
>>
>> (master) vitja at mchome:~/work/cython-vitek-git/zzz$ time python
>> ../cython.py urllib.py
>>
>> real    0m1.764s
>> user    0m1.700s
>> sys     0m0.060s
>
> Ok, then it's only a bug. "create_testscope" is on by default in Main.py,
> Context.__init__(). I don't know what it does exactly, but my guess is that
> the option should a) be off by default and b) should rather be passed in by
> the test runner as part of the compile options rather than being a parameter
> of the Context class. AFAICT, it's currently only used in TreeFragment.py,
> where it is being switched off explicitly for parsing code snippets.
>
> Stefan
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>

It turns it off to avoid infinite recursion. This basically means that
you cannot use stuf from the Cython scope in your Cython utilities. So
in your Cython utilities, you have to declare the C version of it
(which you declared with the @cname decorator).

This is not really something that can just be avoided loading like
this. Perhaps one solution could be to load the test scope when you do
a lookup in the cython scope for which no entry is found. But really,
libcython and serializing entries will solve all this, so I suppose
the real question is, do we want to do a release before we support
such functionality?
Anyway, the cython scope lookup would be a simple hack worth a try.