Map-based module imports with import hook

mpj m.pricejones at gmail.com
Tue Aug 2 06:10:44 CEST 2011


Hi,

I've experience working at companies where, because of the network set
up, having a long PYTHONPATH and searching it is quite a heavy task
and can slow down the start up of the interpreter when there are lots
of imports.

As a proof of concept I wanted to look at a map-based approach. The
theory being that instead of having a linear list of paths to search
with heavy disk checks, if we could encode the location of the modules
in an environment variable we would be able to jump to the correct
location much quicker.

As a result I've written:

    https://github.com/michaeljones/porter

Which implements an import hook (PEP 302) that can be set up to look
at an environment variable with the following format:

    name=location[:name=location[:name=location[:...]]]

This is parsed into a standard dict and used for quick lookups when
possible, falling back to the standard Python import mechanism when
nothing is found.

The implementation is new and not production tested, but I was hoping
to get some external thoughts on the idea, especially to hear any
functionality that this might disrupt. I've only written basic tests
to check import behaviour and am aware that I probably don't really
appreciate the full consequences of this idea.

Cheers,
Michael



More information about the Python-list mailing list