[IPython-dev] Curses Frontend

[Wendell Smith]

Hello,

I have decided to implement a curses frontend to ipython.
My main goals:
  - complete functionality of standard ipython - magics, history, etc.
  - automatic display of completion, not requiring the tab key - tab key 
fills it in
  - automatic display of docstrings, not requiring the '?' key
  - zero pollution of output by docstrings/help (easier to read through 
history)
  - syntax highlighting of input lines as well as output (using 
ipython's color scheme)

The first three will require serious work with curses, as the standard 
library curses module is quite limited, but I believe it can be managed 
in the end with no further dependencies. However, I am wondering if 
perhaps it makes more sense to depend on something like 'urwid'.
As for the fourth, I believe the tokenize module can not handle 
incomplete input, and so I am thinking that it will use 'pygments' if 
its available, and otherwise not highlight. Eventually, perhaps, it 
would have a configuration option for input syntax highlighting; if the 
option were enabled and pygments was not installed, then ipython would 
issue a warning.
I have, of course, seen and used bpython, and this is partially inspired 
from that. However, I cannot see any reasonable way to integrate the two 
projects; while it will be easy to work with ipython, the code for 
bpython does not look... well... easy to apply to ipython, shall we say.

I have a number of questions:
- First and foremost, should it be implemented as an extension, or as a 
frontend + subclass of Shell? I believe implementation as an extension 
is possible, and not terribly complicated or confusing, but it does seem 
a little heavy for the extension system.
- Secondly, how should I handle dependencies? I have not looked into 
anything beyond the main curses library, but using a third-party library 
could quite possibly substantially simplify things, but it would add a 
dependency. Also, I would like input syntax highlighting as noted above, 
and if anyone has comments on that I would welcome suggestions.
- Thirdly, I would like code hosting advice. Should this go in the 
ipython bazaar database? should I get my own something or other? This is 
something I would very much appreciate - let me know how I should handle 
this.
- Lastly, I would like comments on the overall design. My current plan 
is as follows:

------------------------
| completion/help box  |
|    6 lines or so     |
|----------------------|
| ipython output       |
| fills the main space |
| includes completed   |
| input                |
| In [1]: 1            |
| Out[1]: 1             |
|In [2]: 2             |
|Out[2]: 2             |
|----------------------|
|In[3]: | Input box    |
|       | 4 lines or so|
------------------------

My vision is that as one typed, completions  or help would pop-up above. 
If the user pressed tab, completion would fill in as much as possible, 
and also expand to show all the completions available. If one pressed 
tab again, it would page through completions; if one continued typing, 
the completions would narrow; if one finished the word, completions 
would go back to their old 6-10 lines.
If one started a function, or hit ',' within, the function docstring 
would instead show in the upper box, with the same sort of tab-paging 
capability. If one instead used the old '?' and '??' syntax, everything 
would be replaced by a pager, giving one more space to read it all.

I have many more ideas for smaller things - options to output 
python-interpreter-like code on exit, for example - and there will be 
many obstacles to overcome (such as integrating the built-in editor 
capability), I have spent the past week or so going through ipython 
code, considering possibilities, and experimenting, and I think I have a 
fairly good idea of the work ahead of me.

So that is my idea. I have the time and energy to carry it through - I 
know it will take a long time. It will, of course, require at least 
ipython 0.11 - it will almost certainly not be done before that is 
released in any case, and it makes the most sense to use the wonderfully 
reorganized code of 0.11.

Please reply with any comments or suggestions! Also, if anyone is 
already working on something like this, or would like to help me, I 
would love to hear that!

-Wendell