DEF converts byte strings to unicode
DEF silently converts byte strings to Unicode strings: $ cat testcase.pyx DEF ABYTE = b'a' print type(ABYTE) $ python --version Python 2.7.10 $ python -c 'import pyximport as p; p.install(); import testcase' <type 'unicode'> Tested with Cython 0.23.1. -- Jakub Wilk
Jakub Wilk schrieb am 30.08.2015 um 19:44:
DEF silently converts byte strings to Unicode strings:
$ cat testcase.pyx DEF ABYTE = b'a' print type(ABYTE)
$ python --version Python 2.7.10
$ python -c 'import pyximport as p; p.install(); import testcase' <type 'unicode'>
Tested with Cython 0.23.1.
Thanks for the short reproducer. https://github.com/cython/cython/commit/061b132e58812e844ef4e35e6444ba21e0f9... Stefan
I think something is still not quite right in Cython 0.23.2. Consider this code: DEF FOO = 'foo' print type('foo') print type(FOO) In Python 3, I get: <class 'str'> <class 'bytes'> -- Jakub Wilk
Jakub Wilk schrieb am 12.09.2015 um 14:59:
I think something is still not quite right in Cython 0.23.2.
Consider this code:
DEF FOO = 'foo' print type('foo') print type(FOO)
In Python 3, I get:
<class 'str'> <class 'bytes'>
Remember that DEF uses compile time evaluation in *Python*. Python does not have the three string types that Cython has, it has only two: either str/unicode (Py2) or bytes/str (Py3). If you pass an unprefixed string through compile time evaluation, it looses the information that it was unprefixed and turns into a specific Python string object type (i.e. bytes or unicode), which in this case is bytes, lacking any kind of encoding information. Cython follows Py2 semantics by default, so having it turn into a bytes (i.e. Py2 str) object is actually not wrong. Certainly not more wrong than a unicode string would be. If you compile in Py3 mode, you should get a Unicode string. My general recommendation is to a) avoid DEF, b) avoid DEF for string values, and c) avoid DEF for unprefixed string values, in that order. But b) and c) are only for advanced users. Stefan
* Stefan Behnel <stefan_ml@behnel.de>, 2015-09-12, 18:14:
My general recommendation is to a) avoid DEF, b) avoid DEF for string values, and c) avoid DEF for unprefixed string values, in that order. But b) and c) are only for advanced users.
Fair enough. NB, there are examples that use DEFs with string values in the documentation[0]. You might want to fix them, or add appropriate warnings. [0] $ grep -r "DEF.*=.*['\"]" docs/ docs/src/reference/language_basics.rst: DEF FavouriteFood = "spam" docs/src/userguide/language_basics.rst: DEF FavouriteFood = "spam" -- Jakub Wilk
Stefan Behnel schrieb am 12.09.2015 um 18:14:
Jakub Wilk schrieb am 12.09.2015 um 14:59:
I think something is still not quite right in Cython 0.23.2.
Consider this code:
DEF FOO = 'foo' print type('foo') print type(FOO)
In Python 3, I get:
<class 'str'> <class 'bytes'>
Remember that DEF uses compile time evaluation in *Python*. Python does not have the three string types that Cython has, it has only two: either str/unicode (Py2) or bytes/str (Py3). If you pass an unprefixed string through compile time evaluation, it looses the information that it was unprefixed and turns into a specific Python string object type (i.e. bytes or unicode), which in this case is bytes, lacking any kind of encoding information.
Cython follows Py2 semantics by default, so having it turn into a bytes (i.e. Py2 str) object is actually not wrong. Certainly not more wrong than a unicode string would be. If you compile in Py3 mode, you should get a Unicode string.
My general recommendation is to a) avoid DEF, b) avoid DEF for string values, and c) avoid DEF for unprefixed string values, in that order. But b) and c) are only for advanced users.
That being said, always returning a bytes object is actually unhelpful in Python 3. Let's see if anyone complains if we change that. https://github.com/cython/cython/commit/ba350910b67db90c14e0aab79eafe7ac1be1... Stefan
participants (2)
-
Jakub Wilk -
Stefan Behnel