[Python-Dev] PEP 302, PEP 338 and imp.getloader (was Re: a Python interface for the AST (WAS: DRAFT: python-dev...)
ncoghlan at gmail.com
Wed Nov 23 14:51:59 CET 2005
Phillip J. Eby wrote:
> At 06:32 PM 11/22/2005 -0800, Brett Cannon wrote:
>>> Hmm, it would be nice to give a function a module
>>> name (like from an import statement) and have Python resolve it using
>>> the normal sys.path iteration.
>> Yep, import path -> filename path would be cool.
> Zipped and frozen modules don't have filename paths, so I'd personally
> rather see fewer stdlib modules making the assumption that modules are
> files. Instead, extensions to the PEP 302 loader protocol should be used
> to support introspection, assuming there aren't already equivalent
> capabilities available. For example, PEP 302 allows a 'get_source()'
> method on loaders, and I believe the zipimport loader supports that. (I
> don't know about frozen modules.)
> The main barrier to this being really usable is the absence of loader
> objects for the built-in import process. This was proposed by PEP 302, but
> never actually implemented, probably due to time constraints on the Python
> 2.3 release schedule.
> It's relatively easy to implement this "missing loader class" in Python,
> though, and in fact the PEP 302 regression test in the stdlib does exactly
> that. Some work, however, would be required to port this to C and expose
> it from an appropriate module (imp?).
Prompted by this, I finally got around to reading PEP 302 to see how it
related to PEP 338 (which is intended to fix the current limitations of the
'-m' switch by providing a Python fallback when the basic C code can't find
the module to be run).
The key thing that is missing is the "imp.getloader" functionality discussed
at the end of PEP 302.
Using that functionality and the exec statement, PEP 338 could easily be
modified to support any module accessed via a loader which supports get_code()
(and it could probably also get rid of all of the current cruft dealing with
normal filesystem packages).
So with that in mind, I'm thinking of updating PEP 338 to propose the following:
1. A new pure Python module called "runpy"
2. A function called "runpy.execmodule" that is very similar to execfile, but
takes a module reference instead of a filename. It will NOT support
modification of the caller's namespace (based on recent discussions regarding
the exec statement). argv and the name __file__ in the execution dictionary
will be set to the file name for real files (those of type PY_SOURCE or
PY_COMPILED), and the module reference otherwise. An optional argument will
permit argv (and __file__) to be forced to a specific value.**
3. A function called "runpy.get_source" that, given a module reference,
retrieves the source code for that module via loader.get_source()
4. A function called "runpy.get_code" that, given a module reference,
retrieves the code object for that module via loader.get_code()
5. A function called "runpy.is_runnable" that, given a module reference,
determines if execmodule will work on that module (e.g. by checking that the
loader provides the getcode method, that loader.is_package returns false, etc)
6. If invoked as a script, runpy interprets argv as the module to run
7. If the '-m' switch fails to find a module, it invokes runpy as a fallback.
To make PEP 338 independent of the C implementation of imp.getloader for PEP
302 being finished, it would propose two private elements in runpy:
runpy._getloader and runpy._StandardImportMetaHook
If imp.getloader was available, it would be assigned to runpy._getloader,
otherwise runpy would fall back on the Python equivalents.
** I'm open to suggestions on how to deal with argv and __file__. They
should be set to whatever __file__ would be set to by the module loader, but
the Importer Protocol in PEP 302 doesn't seem to expose that information. The
current proposal is a compromise that matches the existing behaviour of -m
(which supports scripts like regrtest.py) while still giving a meaningful
value for scripts which are not part of the normal filesystem.
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Python-Dev