[IPython-dev] Bug caused by commit 2ccc90d68ac01145ab97
Jörgen Stenarson
jorgen.stenarson at bostream.nu
Fri Mar 4 15:00:32 EST 2011
Thomas Kluyver skrev 2011-03-03 21:37:
> On 3 March 2011 19:22, Jörgen Stenarson <jorgen.stenarson at bostream.nu
> <mailto:jorgen.stenarson at bostream.nu>> wrote:
>
> yes it is possible to enter other character codes but you have to
> use the unicode functions in the api to read them properly. It is
> for the 8-bit api that the encoding is valid.
>
>
> OK, I'm not entirely following what you mean here. If I did a =
> raw_input(), and entered a non-cp1252 character, what ends up in a? Does
> it mysteriously turn the output into a unicode object? Or does it
> produce some odd string of bytes?
>
> > In pyreadline we are using the wide api and are returning unicode
> strings, however I have not tested that extensively so there may be bugs
> in that handling.
>
> OK, does that mean that we should add unicode strings to add_history,
> rather than byte strings? Only I think readline in Python 2 on Linux
> choked on unicode when it contained non-ascii characters. Or is there an
> encoding we should use instead of cp1252?
>
> Thomas
Hi,
looking into it in more detail, pyreadline can return unicode strings
but the call the hook that raw_input uses gets a regular string in
return that has been encoded using .encode(pyreadline_codepage, u"replace").
I have attached a script and a screenshot of a session that demonstrates
the difference between using raw_input and calling readline directly.
As things are now, using raw_input you should not receive any characters
outside the current encoding because they will be converted to ? before
being returned to raw_input.
/Jörgen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repl.png
Type: image/png
Size: 4196 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110304/baa76356/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repl.py
Type: application/x-python
Size: 123 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110304/baa76356/attachment.bin>
More information about the IPython-dev
mailing list