[IPython-dev] Towards IPython 1.0, the famous big cleanup

[Fernando Perez]

Hi all,

I'd stopped talking about the famous 'big cleanup' because I was really 
looking like a fool.  But it's really time to get to it.  I have a growing 
list of things I want to do with IPython and which are simply impossible with 
the current structure.

This means that 0.6.13, just released, will most likely be the last release 
for quite a while.  I'm going to be mostly off-line for 2 weeks, which I'll 
consider settling-down time for this release.  If any significant bugs are 
reported in that period, I'll cut a .14 with the fixes, otherwise when I 
return I'll start ripping into the code.

The changes made for Frederic Mantegazza/PyMAD showed at the same time that 
the current code can really be used for fairly sophisticated things, but that 
the architectural limitations make much of this painful.  The fact that it 
looks pretty robust in such an environment makes me comfortable calling a 
'deep freeze' which will probably last several months (ipython is a 'spare 
time' project, after all).

So if you have some really critical bug which is driving you crazy, either 
report it (preferably with a patch) in the next 2-3 weeks, or prepare to live 
with it for quite a while.  The moment I commit the first change to CVS, I 
will simply have to ignore EVERY REQUEST made, until the new shiny ipython 
emerges from the process, reborn in a glory which shall blind the world.  I'm 
sorry to put it this way, but since I do this alone, I simply lack enough 
resources to develop a new codebase while maintaining the current one.

It has been months since a crash has been reported in active code (I do get 
occasional crash reports from users running out-of-date versions).  Because of 
this, I don't think this freeze will be too problematic for anyone.

I should note that in the rewrite, I will make all sorts of incompatible 
changes to pretty much everything.  So if you are a developer of a project 
which uses ipython, unfortunately things will break.  I won't make gratuitous 
changes, but many things in the current system are just too broken to live 
with.  Hopefully the pain will be worth it, as I expect the new system to have 
a far cleaner API for all kinds of customizations and embeddings.  This can be 
done in the current code only because Python allows you to change pretty much 
ANYTHING you want on a running object, not because of any valid design or API. 
  I expect to fix this.

The two most visible changes for embedders and customizers:

- The configuration system will shift from the ad-hoc ipythonrc format to 
proper .py Python source files.  We will thus benefit from the Python compiler 
and engine for code customization, and the configuration loading system will 
be far, far simpler in the end.  I also expect startup times to decrease 
thanks to these changes.

- the magic system will be fully overhauled.  The core of your custom magics 
will most likely run unchanged, but how they plug into ipython will change.


I hope the changes will be worth it, and when the dust settles, we'll have 
something we can call IPython 1.0.  While the current code works quite well, 
I've never wanted to call it 1.0 simply because I consider its (non-existent) 
API to be a prototype.  Once I call something 1.0, I am in a sense committing 
to a reasonably stable public API, which users can rely on (barring gross 
mistakes or bugs) for their own projects in the long run.

As always, I'll welcome any comments, discussion, ideas, patches or help in 
this process.

Regards to all,

Fernando.