[issue28997] test_readline.test_nonascii fails on Android

[Martin Panter]

Martin Panter added the comment:

This is a breakdown of running the test script on my Linux setup (UTF-8 locale):

^A^B^B^B^B^B^B^B\t\tx\t\r\n
    Input echoed back (before Readline disables echo)
[\xc3\xafnserted]
    Inserted by pre_input_hook()
|t\xc3\xab[after]
    Inserted by the Ctrl+A macro
\x08\x08\x08\x08\x08\x08\x08
    Response to moving the cursor left of [after]
text \'t\\xeb\'\r\n
    Encoded completer(text=...) parameter for testing
line \'[\\xefnserted]|t\\xeb[after]\'\r\n
    get_line_buffer() result for testing
indexes 11 13\r\n
    get_begidx(), get_endidx()
\x07
    Rings terminal bell due to multiple possible completions
text . . . line . . . indexes . . .\r\n
    From second round of completer() calls due to second Tab input
substitution \'t\\xeb\'\r\n
matches [\'t\\xebnt\', \'t\\xebxt\']\r\n
    display() parameters
x[after]\x08\x08\x08\x08\x08\x08\x08
    Response to inserting “x”
t[after]\x08\x08\x08\x08\x08\x08\x08
    Completion of "t\xEBxt"
\r\n
    Response to inputting Return
result \'[\\xefnserted]|t\\xebxt[after]\'\r\n
    input() function call return value
history \'[\\xefnserted]|t\\xebxt[after]\'\r\n
    Line as retrieved from history

The problem is that the Ctrl+A macro seems to have only inserted the two ASCII characters "|t" and has stopped at the non-ASCII character "\xEB". Everthing else relies on the macro working properly to get the right cursor positioning and completion text.

Xavier: If you run the following code in an interactive Python session, what does pressing $ print out?

import readline
readline.parse_and_bind('Control-a: "|t\xEB[after]"')
readline.parse_and_bind('"$": dump-macros')

For me, it prints this: (includes “te” with an umlaut)

\C-a outputs |të[after]

What locale encoding does Python use for you? The parse_and_bind() call should be encoding "\xEB" with PyUnicode_EncodeLocale(), and passing the resulting string to Readline. Then Readline is supposed to insert the string when we invoke the macro. Somewhere the string is getting truncated.

----------
nosy: +martin.panter

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28997>
_______________________________________