[IPython-dev] Package-specific extensions: an idea

Robert Kern robert.kern at gmail.com
Tue May 17 18:48:59 EDT 2011

There's a general problem for writing package-specific extensions to IPython. 
Namely, you want to be able to write and enable an extension to IPython that 
provides special behavior for particular objects, say numpy arrays, without 
requiring that a particular package be imported at startup. You can often get by 
in ad hoc ways. For example, in the pretty-printing code, I added the ability to 
register deferred type-checking. For magic functions, you can usually just do 
local imports inside the magics.

However, I've had an idea for a general solution.

Let's say you want to make an extension that does something special with numpy 
arrays. Perhaps it alters the ? introspection to print out array metadata like 
shape and dtype instead of the ndarray docstring when you use ? on arrays. You 
don't want it to modify anything until numpy is imported. You also don't want to 
import numpy at startup every time. You could implement a deferred isinstance() 
check the way I do it in pretty.py, but that's a complicated procedure that 
would need to be replicated everywhere.

Instead, we could have an API for our configuration for deferred extensions, 
namely, a list of (module_name, extension_name) pairs. IPython would keep a 
registry mapping these module names to the extension names. IPython's execution 
loop can be modified to check sys.modules after every execution and activate 
every extension whose requisite modules appear in sys.modules. Then it removes 
those extensions from the list.

So if my extension is named ipy_numpy_oinspect.py, then I'd have something like 
the following in my ipython_config.py file:

c.Global.deferred_extensions = [
     # (module_to_wait_for, extension)
     ('numpy', 'ipy_numpy_oinspect'),

And somewhere in run_cell() after the run_ast() call:

     # Check for any deferred extensions.
     # FIXME: probably wrap this up in a method.
     loaded = []
     for modname, extname in self.extension_manager.deferred_extensions:
         # Explicitly check for is not None since sometimes a missing module
         # will get a None entry to let the import mechanism fail faster.
         if sys.modules.get(modname, None) is not None:
             loaded.append((modname, extname))
     for x in loaded:

Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco

More information about the IPython-dev mailing list