[IPython-dev] Prefilter Reorg Committed

[Dan Milstein]

...finally ;-)

Okay, I just went ahead and committed it to the trunk (r. 2354).

A few notes:

Hey Look, It's a Suite of Tests
===============================

The most useful thing I've got is, IMHO, two fairly extensive suites  
of tests, both in the tests/ directory:

test_prefilter.py
test_handlers.py

Both are, I think, pretty solidly documented.  To run them, you do  
like so from command line:

 > python test_prefilter.py

or

 > python test_handlers.py

(with the relevant version of IPython in your PYTHONPATH).  Notice  
that that is 'python', *not* 'ipython'.

If we can keep those tests up to date, we can go after fairly  
ambitious reorganizations of the input-transformation code with  
decent confidence that things Will Keep Working.

My sincere hope is that it will be easy to add new tests.  If folks  
can take a look at those files and let me know if there is anything  
unclear, I will do my best to make them understandable and  
maintainable.  I have *not* hooked them up to any general suite of  
IPython tests, because I'm not totally clear on if/where that  
exists.  I would love to see them get run as part of a general  
checkin or, at the least, release creation process.



The Basic Structure of the Reorg
================================

I took the really, truly nasty logic from  
iplib.InteractiveShell._prefilter and moved it into a new module,  
prefilter.

As the prefilter module says in its docstring, it is: "responsible,  
primarily, for breaking the line up into useful pieces and triggering  
the appropriate handlers in iplib to do the actual transforming work."

To make much cleanup possible, I collected all the various bits of  
information about a given line (e.g. the raw line itself, the  
preChar, the iFun, etc), into an instance of a new class:  
prefilter.LineInfo.  This makes the interface to the various handlers  
and the stages of the prefilter much, much simpler.

The key stage of prefiltering now looks like so:

     for check in [ checkEmacs,
                    checkIPyAutocall,
                    checkMultiLineShell,
                    checkEscChars,
                    checkPythonChars,
                    checkAutomagic,
                    checkAlias,
                    checkAutocall,
                    ]:
         handler = check(line_info, ip)
         if handler:
             return handler(line_info)

A nice, explicit sequence of checks, which can be understood  
separately.  Each check returns either None or a handler to transform  
the line.


A Random Note
=============

  - I got the new IPyAutocall stuff, and added tests to both  
test_prefilter.py and test_handlers.py to verify the proper behavior


-Dan