[Cython] test failure for cython-devel in Py2.4

[Stefan Behnel]

mark florisson, 13.10.2011 13:52:
> On 13 October 2011 12:44, Stefan Behnel wrote:
>> mark florisson, 13.10.2011 12:18:
>>> On 13 October 2011 10:56, mark florisson wrote:
>>>> On 13 October 2011 10:53, Vitja Makarov wrote:
>>>>> 2011/10/13 Stefan Behnel:
>>>>>> Vitja Makarov, 13.10.2011 08:03:
>>>>>>> I found that tempita bug goes away if you change language_level to 2.
>>>>>>
>>>>>> There's no language level configured in Py2.4, which fails.
>>>>>>
>>>>>>
>>>>>> https://sage.math.washington.edu:8091/hudson/job/cython-devel-tests/48/BACKEND=c,PYVERSION=py24/console
>>>>>
>>>>> No, I mean language level 3 is set at top of the Code.py, when it's
>>>>> set to 2 Py2.4 build is okay.
>>>>>
>>>> Ah, it doesn't take unicode keyword arguments. That should be fixed.
>>>
>>> Frankly, language level 3 is rather uncomfortable to deal with in
>>> python 2(.4).
>>
>> Well, without the parentheses, I presume ...
>
> Ah, it appears only 2.7 eats unicode keyword arguments. I wonder why
> the 2.5 and 2.6 builds didn't fail then.

Ah, right, I remember having to work around this in the C code at some 
point. That's where the "identifier" kind of string in Cython originated from.


>>> Any reason it's set to 3?
>>
>> Mainly for performance reasons, especially in Python 2. Py3 code tends to
>> run faster in Cython due to more explicit semantics. In particular, we get
>> unicode content in and write unicode content out, so using unicode literals
>> in the source code right away saves a decoding step for each write or
>> interpolation of a literal string in Python 2. It won't make a difference
>> when running Cython in Python 3, but it saves a lot of unnecessary
>> processing cycles in Py2, even though the difference may not be substantial
>> over a complete run. It's just so convenient to switch the language level
>> and let that shave off a bunch of processing overhead that I didn't see a
>> reason not to do it.
>>
>> I doubt that it'll make a functional difference, though, so if it works
>> better without that option, we may have to go back to Py2 compilation.
>
> I see. Yeah it's sort of hard to fix, as I really need bytes in python
> 2 and really need unicode (str) in python 3, so I can neither write
> 'foo' nor b'foo' nor u'foo' with language level 3.

You can either pass the keyword arguments explicitly in the code or use 
something like dict(foo=1) to get a dict of keyword arguments (also works 
in Py2.4). Cython will turn the names into identifier strings, i.e. bytes 
in Py2 and unicode in Py3, as required for keyword arguments.


> BTW this is always a real problem in doctests too, as your bytestrings
> will suddenly be printed as b'foo' in python 3, which will fail your
> doctest. So to make it work you need to do explicit encoding/decoding
> to make it work everywhere.

I usually either wrap them in a helper function or, as you say, put a 
.decode() at the end. However, that fails to test explicitly for a byte 
string in Py2, as .decode() also tends to work for ASCII-only unicode 
strings there...

Let's hope that Py2 won't take a decade to die.

Stefan