[IPython-dev] Commits to CVS, fixes, and shell access (RFC)

[Fernando Perez]

Hi all,

I'm trying to catch up a bit on ipython matters.  I just committed some 
bugfixes to CVS for various problems (among others, Solaris crashes caused by 
the curses module and crashes when users closed sys.stdin/out/err).

I also made some 'improvements' to the shell access mechanism, spurred by the 
recent discussions on the matter.   I'll try to say more about that discussion 
in a couple of days, but I'd appreciate some feedback on the changes I made.

Recapping, people have always wanted easy access to the shell, with a way to 
capture the result for further python use.  I added @sc as a magic to do this 
in January, and it's flexible with options.  But at the time, the '!!' syntax 
was also liked, and I realized that a very common usage mode is simply to want 
to loop over shell output in a line-oriented manner (as most unix tools are 
designed).

So I decided today to add (it's in CVS for you to play with) the following:

- @sx, a new magic to execute in the shell any command.  This one, in contrast 
to @sc, doesn't require a variable name.  Instead it _returns_ the output, 
split as a list on '\n', to the normal ipython namespace.  Since ipython 
remembers all output as _N, Out[N], and the last three as _,__ and ___, I 
realized that this might be a very useful thing to have, leveraging the 
existing output storage mechanism.

- !!cmd is a new syntax for '@sx cmd'.  It makes it very easy to get a command 
to the shell remembering and typing as little as possible.

Examples:

In [5]: !ls
IPython-0.5.0.tar.gz  pyamazonpres.tgz

In [6]: sx ls
Out[6]: ['IPython-0.5.0.tar.gz', 'pyamazonpres.tgz']

In [7]: !!ls  # identical to [6] above
Out[7]: ['IPython-0.5.0.tar.gz', 'pyamazonpres.tgz']

In [8]: sc a=ls  # nothing is printed, results in a

In [9]: print a
IPython-0.5.0.tar.gz
pyamazonpres.tgz

In [10]: sc -l a=ls  # results split as a list, then stored in a

In [11]: print a
['IPython-0.5.0.tar.gz', 'pyamazonpres.tgz']


In summary, you can now in ipython access the shell via (in addition to any 
aliases you may have defined):

- !cmd -> execute cmd, return None (output of cmd is printed)

- !!cmd, @sx cmd -> execute cmd and return output of cmd, split on '\n', as a 
python list.

- @sc [options] var=cmd -> execute cmd and store its output in var.  Returns 
None.  Option -l causes a split on '\n' before storing into var, and option -v 
causes the contents to be printed.

@sc is the one with the most flexibility for shell capture, while also 
requiring more typing and remembering options.


I see myself mostly using !cmd as today for quick shell access, and !!cmd when 
I want to pick up for example some filenames for further manipulation through 
python.  !!cmd is a natural enough extension of !cmd that I think I can 
remember it easily.

I'd appreciate feedback on these changes, esp. on whether they enhance 
ipython's functionality for shell access without detracting from its main 
python nature.  I'll try to address more radical (but optional) enhancements 
which others have proposed in another message in a few days.

Also, any reports of breakage (or successful improvements) from the recent CVS 
commits would be very welcome.

Cheers,

f.