[Python-Dev] Write All New Import Hooks (PEP 302) in Python, Not C

Paul Moore lists@morpheus.demon.co.uk
Tue, 31 Dec 2002 20:24:45 +0000


Just van Rossum <just@letterror.com> writes:

> This is all only half true. sys.path_importer_cache is exposed so
> you can *clear* it in case you install a path hook that should take
> over sys.path items that may already have a handler. To only
> partially clear it is advanced usage. By no means you're supposed to
> *add* to sys.path_importer_cache explicitly, as someone after you
> may clear it again, and then you're hosed if there's no proper path
> hook.

Whoo. I was thinking about how I'd simulate the (now unavailable)
feature of adding importers directly to sys.path. It took me virtually
no time at all to come up with

    importer = MyImporter(...)
    imp_id = str(id(importer)) # A cookie
    sys.path.append(imp_id)
    sys.path_importer_cache[imp_id] = importer

The only way I could come up with of doing the same thing without
messing with sys.path_importer_cache was by implementing my own
equivalent - basically

    id_cache = {}
    def id_factory(id):
        try:
            return id_cache[id]
        except KeyError:
            raise ImportError
    sys.path_hooks.append(id_factory)

    importer = MyImporter(...)
    imp_id = str(id(importer)) # A cookie
    sys.path.append(imp_id)
    id_cache[imp_id] = importer

I can't think of any reason why this is *intrinsically* better than
the first way other than because someone may clear the cache on
you. And it's certainly not as intuitive.

> I also disagree that adding an importer to sys.path_importer_cache
> would always be *easier* than doing it the right way by adding a
> hook to sys.path_hooks.

Hmm. See above. OK, so that's generic code, and specific cases may
always be easier to code "the right way".

> If would be somewhat easier for "cookie" path items, but I think in
> 99% of these cases a path-less importer on sys.meta_path would work
> just as well.

But sys.meta_path happens *before* sys.path. It's entirely likely that
a cookie item might want to run *after* sys.path. For that we need
"stage 2", where the real sys.path importer is a proper item on
sys.meta_path.

At the moment, whether sys.meta_path should run before or after
sys.path is a judgement call - which usage is more common? In the
longer term, the normal insert(0) vs append() approach gives both
options.

Disclaimer: As usual, this is all theory. Real use cases take
precedence.

Paul.

PS Jim's comments have now raised two (as far as I can recall) issues
   with the proposed hook mechanism which should be documented in the
   "Open Issues" section of the PEP:

   1. sys.meta_path allows hooks to be added *before* sys.path
      processing, but there is no equivalent facility for hooks
      *after* sys.path (response: either wait for "phase 2" or use a
      cookie at the end of sys.path).
   2. There is no easy way of "stacking" hooks (response: there isn't
      one now, either, so that's not a critical issue).
-- 
This signature intentionally left blank