[Tutor] ipython / readline problem

[Tiago Saboga]

Hi,

I am facing what seems like a bug, and the docs are not enough to tell
me where the bug lies. In ipython, I can't type a multiline unicode
string. The bug is reported here [1], and the output is like that
(except that the commented out lines are modifications I have done):

===================================
In [1]: a = '''ç
   ...: '''
---------------------------------------------------------------------------
<type 'exceptions.UnicodeEncodeError'>    Traceback (most recent call last)

/var/lib/python-support/python2.5/IPython/iplib.py in raw_input(self, prompt, continue_prompt)
   2041                         self.readline.remove_history_item(histlen-1)
   2042                         # newhist = newhist.encode('ascii', 'replace')
-> 2043                         self.readline.replace_history_item(histlen-2,newhist)
   2044                         #self.input_hist_raw[-1] = newhist
   2045                     except AttributeError:

<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode character u'\xe7' in position 7: ordinal not in range(128)

===================================

So I went to the ipython source code to see what was wrong and finally
I discovered that one can't pass unicode strings to readline. To be
clearer, the behaviour above is found only in ipython, but the errors
below can be reproduced also in the python console:

===================================

In [6]: readline.add_history('ação')

In [7]: readline.get_current_history_length()
Out[7]: 1008

In [8]: readline.replace_history_item(1008, 'ação')

In [9]: readline.replace_history_item(1008, u'ação')
---------------------------------------------------------------------------
<type 'exceptions.UnicodeEncodeError'>    Traceback (most recent call
last)

/home/tiago/<ipython console> in <module>()

<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode
characters in position 1-2: ordinal not in range(128)

In [10]: readline.add_history(u'caça')
---------------------------------------------------------------------------
<type 'exceptions.UnicodeEncodeError'>    Traceback (most recent call
last)

/home/tiago/<ipython console> in <module>()

<type 'exceptions.UnicodeEncodeError'>: 'ascii' codec can't encode
character u'\xe7' in position 2: ordinal not in range(128)

=======================================

What's happening? Why do the readline methods accept a multibyte
string ('ação') but not a unicode (u'ação')?

Thanks,

Tiago.

[1] - http://projects.scipy.org/ipython/ipython/ticket/201