[IPython-dev] Configuration, hooks and plugins

[Brian Granger]

Hi,

One of the things I am going to do in refactoring the IPython core
this summer is to consolidate the current configuration and hooks
system.

As I see it, here is our current situation.  We have 3 different
configuration systems:

1.  The basic config stuff in ipythonrc.  This uses key/value pairs in the form:

key value

2.  The more dynamic stuff in ipy_user_conf.py

This allows the user to dynamically configure ipython after it is
running by calling ipapi, loading extensions and setting hooks.

3.  IPython.kernel uses .ini files for configuring the parallel computing stuff.

This we current have:  3 config systems and hooks.  I propose that we
move to 1 config system and a proper plugin system (maybe just a
different name for a hook).

Config System
===========

By a "config system" I mean key, value pairs that determine how
IPython classes are created and behave.  But, it is a bit more
complicated than this.  First, some types of keys/values should be
changed only when IPython is starting up, other types of keys can be
changed at any time while IPython is running (such as the traceback
type, color mode, etc).  Also, some types of key/values can be set at
the command line as well.  Thus, here are some characteristics that
our config system needs to have:

* KISS.  A pure python config system that can be implemented in a few
hours, not weeks or months.  If we use an external dependency it must
be trivial to package and pure python (no traits!)

* An IPython module/class/function needs to be able to say "here is a
configuration key, here is it's default value, it will appear is this
config file, here is how it should show up at the command line, it
should/shouldn't be changed dynamically"

* When objects are created, they need to be able to get the current
global value of the configuration key.  This raises the question of
how configuration information gets to IPython objects.

* We need a way of handling things that can be set at startup, but
that can also be changed at runtime (often through magics).  We might
way to keep these things separate though.  We could say that "config"
means things that are set when an object is created and can't be
changed from then on.  To handle things that can be changed at
runtime, we could have a completely different api that ties into the
magic system.

Plugins
----------

Currently we use "hooks" to allow people to customize how IPython
behaves in non-trivial ways.  Typically, hooks involve implementing
some complex behavior by in a python function.  The current hooks
system is quite messy and grew into existence partly because IPython
simply didn't use good object oriented design.  I propose that we move
to a plugin system, where a plugin is a python class that adheres to a
very narrow and API and implements a single behavior.  We should have
a very simple API for registering plugins at startup and runtime, and
we probably want to create a .ipython/plugins directory where users
can put plugins.  Many of the things in Extensions will probably end
up as plugins.  I have fewer thoughts on how to implement plugins, but
I know that we won't be able to use any of the fancy ways of doing
plugins like zope or setuptools.

At this point, I am looking for feedback, ideas and discussions about
these things.

Cheers,

Brian