[Python-Dev] PEP 302 support for pydoc, runpy, etc.
Phillip J. Eby
pje at telecommunity.com
Fri Apr 14 05:33:36 CEST 2006
Okay, so I've spent a bit more time thinking this through, and I think I
have a plan that can work, and deals with all the weird little corner
issues, including overlapping code with pkg_resources (a new module that's
to-be-added by setuptools).
My basic idea is as follows:
1. Move the support code from runpy to pkgutil and expose most of it as
2. Move additional code from pkg_resources to pkgutil and merge with the
stuff brought over from runpy, fleshing it out enough so that pydoc can use
it to find modules, subpackages, etc., and so that pkg_resources' other
code uses pkgutil instead of having that stuff "in-house".
3. Write up docs for the expanded pkgutil
4. Add find_loader([path]), get_loader(module), and
get_importer(string[,path_hooks]) functions to the 'imp' module, with
pure-Python versions in pkgutil that kick in if the C ones are
missing. (See next item for why.)
5. Distribute the Python 2.5 'pkgutil' module with setuptools releases for
older Python versions, so that setuptools won't have to keep duplicates of
the code I pulled out in item #2 above.
Part of what would be being copied to pkgutil from pkg_resources is the
"resource provider API", that allows you to treat a module, package, or
sys.path entry as a kind of virtual filesystem. pydoc needs this in order
to be able to document with zipped packages, and to find modules or
packages on sys.path that are in zipfiles. (E.g. to document a zipped
standard library package or module.)
pkg_resources has been out there for almost a year now, and is widely used
as part of easy_install and the egg runtime facilities, but it's a very
large module and contains lots of other code besides the virtual filesystem
stuff. So, this seems like the simplest way to share the virtual
filesystem code between pydoc and setuptools, without making the former
depend on the latter.
It does also mean creating an addition to PEP 302: an optional method on
importers and loaders that would allow obtaining a virtual filesystem for
that path location or package.
I believe this and the setuptools checkin can be completed by Monday
evening, although the documentation might not be fancy at that point. I
don't know when the documentation cutoff for alpha 2 is, but I assume I'd
have time to finish fleshing it out, at least for the pkgutil bit and the
Anybody have any thoughts, comments, or questions?
More information about the Python-Dev