[Cython] CPython incompatibility in string literal concatenation

[Stefan Behnel]

Hi,

sorry for the late reply.

Jelle Zijlstra schrieb am 13.09.2015 um 21:00:
> While implementing PEP 498 I found a minor incompatibility between
> CPython and Cython: CPython lets you concatenate u-prefixed and
> non-prefixed string literals, while Cython throws an error.
> 
> jelle at devjelle:~/cython$ cat concat.py
> print(u'foo' 'bar')
> 
> jelle at devjelle:~/cython$ python concat.py
> foobar
> jelle at devjelle:~/cython$ cython concat.py
> 
> Error compiling Cython file:
> ------------------------------------------------------------
> ...
> print(u'foo' 'bar')
>             ^
> ------------------------------------------------------------
> 
> concat.py:1:13: Cannot mix string literals of different types,
> expected u'', got ''
> 
> 
> I tried both CPython 2.7 and 3.5. The documentation at
> https://docs.python.org/2/reference/lexical_analysis.html#string-literal-concatenation
> is ambiguous as to whether this behavior is intentional.
> 
> As part of implementing PEP 498, I can make Cython's behavior match
> CPython's, unless there's a good reason to keep the incompatibility.

This obviously works in Py3.3+, where the "u" prefix is simply ignored. It
also works in Py2 because of the implicit promotion from byte strings to
unicode strings.

I'm ok with supporting this in Cython. The string literal parser already
generates a Unicode representation of unprefixed strings which can be used
for concatenation. In fact, the above is currently an explicit error case.

Stefan