[IPython-dev] Fairly deep changes in CVS, testers welcome

[Fernando Perez]

Hi all,

I know I recently put out a 'release candidate', but please bear with me. 
I've committed a raft of pretty serious changes which do touch the critical 
inner loops in ipython, so I'm a bit worried and would really like some feedback.

I hope to lure you in because the offered functionality is somewhat 
interesting, IMHO.  I finally got around to rewriting the whole ipython alias 
subsystem.  The old one was based on a trick which felt neat when I was in my 
2nd week of python learning (dynamic code generation and runtime modification 
of the running instance method table).  In reality it was just a heavyweight 
hack, which could be replaced by a simple, light dictionary.

With the new code in place, we now have:

In [1]: len(__IP.alias_table)
Out[1]: 24

In [2]: rehash  # takes ~ 0.3 s to run on my laptop

In [3]: len(__IP.alias_table)
Out[3]: 3652

rehash is a new magic which works just like the one in tcsh.  Which means we 
can now do things like:

In [4]: python
Python 2.2.3 (#1, Oct 15 2003, 23:33:35)
[GCC 3.3.1 20030930 (Red Hat Linux 3.3.1-6)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>>

In [5]: latex
This is TeX, Version 3.14159 (Web2C 7.4.5)
**
! End of file on the terminal... why?

and:

In [6]: PATH='a normal python string'

In [7]: echo $PATH
a normal python string

In [8]: echo $$PATH
/usr/local/lf9560/bin:/usr/local/intel/compiler70/ia32/bin: ... [SNIP]

the last two rely on new escaping functionality I also introduced.  You can 
now call system commands (whether aliases or via !, !!, @sc and @sx) and 
expand any python expression you want (see PEP-215 at 
http://www.python.org/peps/pep-0215.html for details).  A double $$ will pass 
a single $ to the shell so you can also access normal shell vars.

All of this means that now pysh (my alias for 'ipython -p shell') is a much 
more realistic environment as a shell.  In particular, I rewrote the readline 
support so that it's now trivial to define the order of priorities for tab 
completion.  The orders are now:

- pysh matchers = ['file_matches','alias_matches','python_matches']

- mormal matchers = ['python_matches','file_matches','alias_matches']

Run pysh (well, 'ipython -p shell') and let me know how it feels, and how it 
actually performs.

The only really major thing missing from pysh to be usable as a light shell (I 
don't claim it to be in the same league as bash/tcsh) is that all of this 
magic only applies for single-line statements.  If you want to write 
multi-line code, you still need to call system() yourself and do your own 
variable expansions.

I actually don't see this as a big drawback:  this case is much less common, 
and if you are already typing multiple lines, being a bit explicit on exactly 
what you want is probably a good thing.  I only wanted the 'quick and dirty' 
single-line statements to be as expressive as possible.  As always, ipython 
tries to give as much 'punch per keystroke' as it can.

In summary, the new goodies are:

- Python variable expansion in all system access mechanisms (!, !!, @sc, @sx 
and aliases).

- Lightweight alias table and new @rehash/@rehashx magics for pulling all of 
$PATH as aliases.

- shell profile which uses these (it starts with a @rehash), and reorders 
tab-completion so files come first.

I'm going to let these ideas simmer in CVS for a while before pushing them 
further.  The changelog has more details, as usual.  And Viktor Ransmayr is 
working hard on the nasty distutils-windows battle, so I want to give that 
time to mature as well.

In particular, @rehash/x are very Unix-specific.  I don't know how to deal 
with $PATH and how to make executability decisions under Windows, which (I 
think) has a baroque extension-based mechanism for this.  If someone else 
wants to write this code, look at Magic.py for the Unix examples.


I would very much appreciate feedback, both of success and (hopefully not too 
much) of problems.  As I said, these ideas forced me to really cut in places I 
normally try to stay away from, so there's a chance I could have broken 
something big.

Best,

f