[ python-Bugs-1009263 ] non-ascii readline input crashes python

[SourceForge.net]

Bugs item #1009263, was opened at 2004-08-14 19:19
Message generated for change (Comment added) made by mwh
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1009263&group_id=5470

Category: None
Group: None
>Status: Closed
>Resolution: Out of Date
Priority: 5
Submitted By: paul rubin (phr)
Assigned to: Nobody/Anonymous (nobody)
Summary: non-ascii readline input crashes python

Initial Comment:
Start Python 2.3.4 on Linux command line:

$ python
Python 2.3.4 (#1, Jul 24 2004, 19:45:58)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>>

Enter the string "2×3" (2 times 3, the symbol between 2
and 3 is a multiplication sign, I think Unicode c397 or
maybe 97c3 if I have it byte swapped) and hit return. 
Python reports a syntax error.  Hit Ctrl-P to edit the
line with readline.  Use ctrl-D to delete the first
character of the input line, and Python instantly
crashes with a seg fault.

----------------------------------------------------------------------

>Comment By: Michael Hudson (mwh)
Date: 2005-04-07 10:07

Message:
Logged In: YES 
user_id=6656

Closing for want of response.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-09-07 17:02

Message:
Logged In: YES 
user_id=6656

ping

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-08-16 13:29

Message:
Logged In: YES 
user_id=6656

You should check the $LANG or $LC_ALL env vars.  Or just run
'locale' in the shell.

What version of readline are you using?  Both instinct, and
looking at the backtrace, suggest a readline bug to me.

----------------------------------------------------------------------

Comment By: paul rubin (phr)
Date: 2004-08-15 02:55

Message:
Logged In: YES 
user_id=72053

Backtrace follows.  I don't know how to tell my locale.  My
LOCALE shell variable is not set to anything--is there
something else I need to check?  Since the "2×3" displays
properly in my terminal window, I think I'm likely using UTF-8.


(gdb) r
Starting program: /usr/local/bin/python
[New Thread 1074951328 (LWP 6561)]
Python 2.3.4 (#1, Jul 24 2004, 19:45:58)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
Type "help", "copyright", "credits" or "license" for more
information.
>>> 2×3
  File "<stdin>", line 1
    2×3
     ^
SyntaxError: invalid syntax
>>> 2×3
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1074951328 (LWP 6561)]
0x401eb687 in _rl_get_char_len () from /usr/lib/libreadline.so.4
(gdb) bt
#0  0x401eb687 in _rl_get_char_len () from
/usr/lib/libreadline.so.4
#1  0x401eb701 in _rl_compare_chars () from
/usr/lib/libreadline.so.4
#2  0x401e0030 in rl_redisplay () from /usr/lib/libreadline.so.4
#3  0x401de816 in rl_redisplay () from /usr/lib/libreadline.so.4
#4  0x401d2588 in readline_internal_char () from
/usr/lib/libreadline.so.4
#5  0x401d2695 in readline_internal_char () from
/usr/lib/libreadline.so.4
#6  0x401d26ce in readline_internal_char () from
/usr/lib/libreadline.so.4
#7  0x401d219f in readline () from /usr/lib/libreadline.so.4
#8  0x40019074 in call_readline (sys_stdin=0x4212fcc0,
sys_stdout=0x4212fe20,
    prompt=0x40177254 ">>> ")
    at /home/phr/python/Python-2.3.4/Modules/readline.c:679
#9  0x080e6110 in PyOS_Readline (sys_stdin=0x4212fcc0,
sys_stdout=0x4212fe20,
    prompt=0x40177254 ">>> ") at Parser/myreadline.c:183
#10 0x08056c47 in tok_nextc (tok=0x817f428) at
Parser/tokenizer.c:683
#11 0x08055d8f in tok_get (tok=0x817f428,
p_start=0xbfffecc4, p_end=0xbfffecc8)
    at Parser/tokenizer.c:1014
#12 0x080559de in PyTokenizer_Get (tok=0x817f428,
p_start=0xbfffecc4,
    p_end=0xbfffecc8) at Parser/tokenizer.c:1414
#13 0x0805543a in parsetok (tok=0x817f428, g=0x8126b88,
start=256,
    err_ret=0xbfffed10, flags=0) at Parser/parsetok.c:125
#14 0x080cb0e2 in PyRun_InteractiveOneFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", flags=0xbfffedf8) at
Python/pythonrun.c:738
#15 0x080caf3b in PyRun_InteractiveLoopFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", flags=0xbfffedf8) at
Python/pythonrun.c:690
#16 0x080cc2ee in PyRun_AnyFileExFlags (fp=0x4212fcc0,
    filename=0x80f004a "<stdin>", closeit=0, flags=0xbfffedf8)
    at Python/pythonrun.c:653
#17 0x08054d32 in Py_Main (argc=0, argv=0xbfffee74) at
Modules/main.c:415
#18 0x080548ab in main (argc=1, argv=0xbfffee74) at
Modules/python.c:23
#19 0x420156a4 in __libc_start_main () from /lib/tls/libc.so.6


----------------------------------------------------------------------

Comment By: Hye-Shik Chang (perky)
Date: 2004-08-14 22:10

Message:
Logged In: YES 
user_id=55188

What locale do you use?  I couldn't reproduce the problem on
any of utf-8, iso8859-1, C, euc-kr locale.
And, can you please attach a backtrace for the segfault point?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1009263&group_id=5470