Plugins in Python

[David Bolen]

David LeBlanc <whisper at oz.nospamnet> writes:

> Thanks David! That's exactly what i'm looking for. Python the ever 
> flexible snake! I sort of thought that (dunno why, just didn't occur to 
> me) that import wouldn't take a variable as in:
> 
> mymodule = "c:/boo/yikes.py"
> import mymodule
> 
> I gather that's what you mean?

Close - import in fact doesn't take such a variable, but you can use
__import__ instead and give it a string.

As two quick samples, below are two __init__ files (taken directly
from a remote administration tool project of mine, so not all comments
may make sense :-)).  The first runs remotely at a site, and doesn't
get packaged up but directly distributed, so I let the __init__
determine completely dynamically what to import.  I just push out new
modules into the package directory to add to the system.  I go a
little further in this one and borrow DateTime's LazyModule to hold
off the actual import until the system wants to use the plugin.  If
you wanted a direct import, it's a little easier as you don't have to
worry about the package path, just use:

    __import__('<modulename>',globals(),locals())

The second is used in my UI tool that ops folks run, so it does
sometimes get packaged up, and here I've decided to explicitly import
the major plugins, but dynamically determine what they offer at
runtime, which the UI tool then uses to determine what "pages" in the
tool to display.

---- Dynamic package ----

"""\
nimgmt package

This is a package of modules for remote management of sites, designed
to be used both on a site via an nimgmtd.py driver module, or centrally
as part of a GUI site management tool.

When imported, a supported "modules" dictionary is populated for the
individual Python modules discovered in the package, and those modules are
set up for lazy loading on a subsequent reference.  The dictionary contains
a module reference (initially set to be lazy loaded) for each module.

The support modules are determined dynamically from the contents of the
package directory, excluding files starting with "_".

The get_object() function may be used to retrieve an instantiated management
object for a module (imported through the lazy definition) - it acts as a
singleton factory function, caching a module's object once created.
"""

#
# ----------------------------------------------------------------------
#

class Error(Exception):
    """General nimgmt exception class available to all modules"""
    pass

#
# ----------------------------------------------------------------------
#

def find_modules():
    import sys, os, fnmatch, string
    import DateTime.LazyModule

    LazyModule = sys.modules['DateTime.LazyModule']

    # Walk the current directory to identify all management modules,
    # and store a reference to them within the returned dictionary.

    modfiles = filter(lambda x,m=fnmatch.fnmatch:m(x,'*.py'),
                      os.listdir(__path__[0]))

    modules    = {}

    for curmodule in modfiles:
        modname = os.path.splitext(curmodule)[0]
        if modname[0] == '_':
            continue
        else:
            modules[modname] = {'module':
                                LazyModule.LazyModule(__name__+'.'+modname,
                                                      locals(),globals())}

    return modules

#
# ----------------------------------------------------------------------
#

modules = find_modules()

del find_modules


---- Semi-static package ----

"""Package of "plug-in" modules for the NI Site Administration Utility.

Each of the modules in this package augment the stock administration tool
by managing one or more of the administration "pages" to add functionality.

Upon import, this package file creates a package-level classes dictionary,
indexed by group name, the value of which is a dictionary indexed by page
name.  Each page value is a dictionary containing attributes of that page,
currently:

    class       - wxWindow derived class to instantiate for UI display
    description - Brief description of purpose of that class

For a Python module within this package directory be suitable for inclusion
in the classes dictionary, it must have the following required attribute:

    NIMgmt()    - Return nested dictionary, keyed by category and then
                  operation, of the same structure as described above
                  containing information on any classes supported by that
                  module.
"""

import sys

#
# ----------------------------------------------------------------------
#  
#  Import each main plugin module.  This is done manually (requiring
#  this file to be kept up to date as new plugins are added) to avoid
#  a lot of extra work later if packaging up this utility for the
#  installer package, which can't follow dynamic importing.  Only the
#  module has to be imported, as the local dictionary is automatically
#  scanned for imported modules that have the administrative function
#  used to return information for populating the modules dictionary.
#
# ----------------------------------------------------------------------
#  

import Help
import Inventory
import EventLog
import Services
import Cardio
import Strength
import Local
import Demo
import Disk
import Grep

classes = {}

#
# ----------------------------------------------------------------------
#
#  Now walk modules we've imported and generate other information based
#  on the contents of those modules.
#
# ----------------------------------------------------------------------
#

for module in globals().values():
    if type(module) == type(sys) and module.__dict__.has_key('NIMgmt'):
        try:
            module_classes = module.NIMgmt()
            # XXX: Sanity check this to ensure all attributes available?
            #      This doesn't protect against per-tab overwriting
            for group in module_classes.keys():
                if not classes.has_key(group):
                    classes[group] = {}
                classes[group].update(module_classes[group])
        except:
            # Bad module - just skip
            # XXX: Log this better?
            pass

del module, group

> I take your point about the installer issue, and it's something i'd have 
> to live with I guess. I want to be able to dynamically download a new 
> plugin and have the user take no further action then putting it in the 
> plugins directory. This might even be done automagically as part of a 
> dynamic update or install feature and then the user couldn't even get 
> putting it in the right directory wrong ;-)

Well, you could also do a mixture.  Have the main application fully
installable (built with py2exe or installer), but leave the plugins
subdirectory plain, and just something included as part of the overall
installation package (say with Inno Setup).  Then the dynamic approach
should be fine and you can drop in any file (or build an install tool
that just drops in the file).

Of course, even if you go with the more explicit approach, all that
means is that distributing a new plugin is two files - the plugin
module and the updated __init__.  If different users may have
different plugins, just always distribute the latest __init__ and
protect against missing plugins with a try/except block around the
import.

And there's probably a ton of other ways to slice it.


--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/