[Python-checkins] peps: [PEP 451] Update the signature of find_spec() and remove supports_reload().
eric.snow
python-checkins at python.org
Fri Nov 1 23:23:26 CET 2013
http://hg.python.org/peps/rev/845e08302be8
changeset: 5244:845e08302be8
user: Eric Snow <ericsnowcurrently at gmail.com>
date: Fri Nov 01 16:19:06 2013 -0600
summary:
[PEP 451] Update the signature of find_spec() and remove supports_reload().
files:
pep-0451.txt | 238 +++++++++++++++++++++++++-------------
1 files changed, 156 insertions(+), 82 deletions(-)
diff --git a/pep-0451.txt b/pep-0451.txt
--- a/pep-0451.txt
+++ b/pep-0451.txt
@@ -278,9 +278,9 @@
Other API Additions
-------------------
-* importlib.find_spec(name, path=None) will work exactly the same as
- importlib.find_loader() (which it replaces), but return a spec instead
- of a loader.
+* importlib.find_spec(name, path=None, existing=None) will work exactly
+ the same as importlib.find_loader() (which it replaces), but return a
+ spec instead of a loader.
For finders:
@@ -295,8 +295,6 @@
over its module execution functionality.
* importlib.abc.Loader.create_module(spec) (optional) will return the
module to use for loading.
-* importlib.abc.Loader.supports_reload(name) (optional) will return True
- (the default) if the loader supports reloading the module.
For modules:
@@ -374,13 +372,13 @@
finders:
-* create loader
-* create spec
+* create/identify a loader that can load the module.
+* create the spec for the module.
loaders:
-* create module (optional)
-* execute module
+* create the module (optional).
+* execute the module.
ModuleSpec:
@@ -404,11 +402,9 @@
* Implement exec_module() on loaders, if possible.
The ModuleSpec factory functions in importlib.util are intended to be
-helpful for converting existing finders. from_loader() and
-from_file_location() are both straight-forward utilities in this
-regard. In the case where loaders already expose methods for creating
-and preparing modules, ModuleSpec.from_module() may be useful to
-the corresponding finder.
+helpful for converting existing finders. spec_from_loader() and
+spec_from_file_location() are both straight-forward utilities in this
+regard.
For existing loaders, exec_module() should be a relatively direct
conversion from the non-boilerplate portion of load_module(). In some
@@ -497,7 +493,7 @@
name = module.__spec__.name
except AttributeError:
name = module.__name__
- spec = find_spec(name)
+ spec = find_spec(name, existing=module)
if sys.modules.get(name) is not module:
raise ImportError
@@ -509,8 +505,6 @@
# namespace loader
_init_module_attrs(spec, module)
return module
- if not spec.loader.supports_reload(name):
- raise ImportError
if spec.parent and spec.parent not in sys.modules:
raise ImportError
@@ -520,6 +514,19 @@
finally:
del _RELOADING[name]
+A key point here is the switch to Loader.exec_module() means that
+loaders will no longer have an easy way to know at execution time if it
+is a reload or not. Before this proposal, they could simply check to
+see if the module was already in sys.modules. Now, by the time
+exec_module() is called during load (not reload) the import machinery
+would already have placed the module in sys.modules. This is part of
+the reason why find_spec() has
+`the "existing" parameter <The "existing" parameter of find_spec()>`_.
+
+The semantics of reload will remain essentially the same as they exist
+already [reload-semantics-fix]_. The impact of this PEP on some kinds
+of lazy loading modules was a point of discussion. [lazy_import_concerns]_
+
ModuleSpec
==========
@@ -627,7 +634,7 @@
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if location is a filename.
-**from_loader(name, loader, \*, origin=None, is_package=None)**
+**spec_from_loader(name, loader, \*, origin=None, is_package=None)**
Build a spec with missing information filled in by using loader APIs.
@@ -636,45 +643,6 @@
* "submodule_search_locations" can be deduced from loader.is_package()
and from os.path.dirname(location) if location is a filename.
-**spec_from_module(module, loader=None)**
-
-Build a spec based on the import-related attributes of an existing
-module. The spec attributes are set to the corresponding import-
-related module attributes. See the table in `Attributes`_.
-
-Omitted Attributes and Methods
-------------------------------
-
-There is no "PathModuleSpec" subclass of ModuleSpec that separates out
-has_location, cached, and submodule_search_locations. While that might
-make the separation cleaner, module objects don't have that distinction.
-ModuleSpec will support both cases equally well.
-
-While "is_package" would be a simple additional attribute (aliasing
-self.submodule_search_locations is not None), it perpetuates the
-artificial (and mostly erroneous) distinction between modules and
-packages.
-
-Conceivably, a ModuleSpec.load() method could optionally take a list of
-modules with which to interact instead of sys.modules. That
-capability is left out of this PEP, but may be pursued separately at
-some other time, including relative to PEP 406 (import engine).
-
-Likewise load() could be leveraged to implement multi-version
-imports. While interesting, doing so is outside the scope of this
-proposal.
-
-Others:
-
-* Add ModuleSpec.submodules (RO-property) - returns possible submodules
- relative to the spec.
-* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
- any.
-* Add ModuleSpec.data - a descriptor that wraps the data API of the
- spec's loader.
-* Also see [cleaner_reload_support]_.
-
-
Backward Compatibility
----------------------
@@ -722,15 +690,16 @@
Finders
-------
-Finders are still responsible for creating the loader. That loader will
+Finders are still responsible for identifying, an typically creating,
+the loader that should be used to load a module. That loader will
now be stored in the module spec returned by find_spec() rather
than returned directly. As is currently the case without the PEP, if a
loader would be costly to create, that loader can be designed to defer
the cost until later.
-**MetaPathFinder.find_spec(name, path=None)**
+**MetaPathFinder.find_spec(name, path=None, existing=None)**
-**PathEntryFinder.find_spec(name)**
+**PathEntryFinder.find_spec(name, existing=None)**
Finders must return ModuleSpec objects when find_spec() is
called. This new method replaces find_module() and
@@ -745,6 +714,42 @@
added in Python 3.3. However, the extra complexity and a less-than-
explicit method name aren't worth it.
+The "existing" parameter of find_spec()
+---------------------------------------
+
+A module object with the same name as the "name" argument (or None, the
+default) should be passed in to "exising". This argument allows the
+finder to build the module spec with more information than is otherwise
+available. This is particularly relevant in identifying the loader to
+use.
+
+Through find_spec() the finder will always identify the loader it
+will return in the spec. In the case of reload, at this point the
+finder should also decide whether or not the loader supports loading
+into the module-to-be-reloaded (which was passed in to find_spec() as
+"existing"). This decision may entail consulting with the loader. If
+the finder determines that the loader does not support reloading that
+module, it should either find another loader or return None (indicating
+that it could not "find" the module). This reload decision is important
+since, as noted in `How Reloading Will Work`_, loaders will no longer be
+able to trivially identify a reload situation on their own.
+
+Two alternatives were presented to the "existing" parameter:
+Loader.supports_reload() and adding "existing" to Loader.exec_module()
+instead of find_spec(). supports_reload() was the initial approach to
+the reload situation. [supports_reload]_ However, there was some
+opposition to the loader-specific, reload-centric approach.
+[supports_reload_considered_harmful]_
+
+As to "existing" on exec_module(), the loader may need other information
+from the existing module (or spec) during reload, more than just "does
+this loader support reloading this module", that is no longer available
+with the move away from load_module(). A proposal on the table was to
+add something like "existing" to exec_module(). [exec_module_existing]_
+However, putting "existing" on find_spec() instead is more in line with
+the goals of this PEP. Furthermore, it obviates the need for
+supports_reload().
+
Namespace Packages
------------------
@@ -791,13 +796,6 @@
module attributes. The fact that load_module() does is a design flaw
that this proposal aims to correct.
-**Loader.supports_reload(name)**
-
-In cases where a module should not be reloaded, Loaders should implement
-supports_reload() and have it return False. If the method is defined
-and returns a false value, importlib.reload() will raise an ImportError.
-Otherwise reloading proceeds as normal.
-
Other changes:
PEP 420 introduced the optional module_repr() loader method to limit
@@ -837,24 +835,27 @@
* importlib.reload() will now make use of the per-module import lock.
+Open Issues
+===========
+
+* In the `Finders`_ section, the PEP specifies returning None (or using
+a different loader) when the found loader does not support loading into
+an existing module (e.g during reload). An alternative to returning
+None would be to raise ImportError with a message like "the loader does
+not support reloading the module". This may actually be a better
+approach since "could not find a loader" and "the found loader won't
+work" are different situations that a single return value (None) may not
+sufficiently represent.
+
+
Reference Implementation
========================
-A reference implementation will be available at
+A reference implementation is available at
http://bugs.python.org/issue18864.
-
-Open Issues
-==============
-
-\* Impact on some kinds of lazy loading modules. [lazy_import_concerns]_
-
-This should not be an issue since the PEP does not change the semantics
-of this behavior.
-
-
Implementation Notes
-====================
+--------------------
\* The implementation of this PEP needs to be cognizant of its impact on
pkgutil (and setuptools). pkgutil has some generic function-based
@@ -868,17 +869,90 @@
at ``module.__spec__.name``.
+Rejected Additions to the PEP
+=============================
+
+There were a few proposed additions to this proposal that did not fit
+well enough into its scope.
+
+There is no "PathModuleSpec" subclass of ModuleSpec that separates out
+has_location, cached, and submodule_search_locations. While that might
+make the separation cleaner, module objects don't have that distinction.
+ModuleSpec will support both cases equally well.
+
+While "ModuleSpec.is_package" would be a simple additional attribute
+(aliasing self.submodule_search_locations is not None), it perpetuates
+the artificial (and mostly erroneous) distinction between modules and
+packages.
+
+Others left out:
+
+* Add ModuleSpec.submodules (RO-property) - returns possible submodules
+ relative to the spec.
+* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
+ any.
+* Add ModuleSpec.data - a descriptor that wraps the data API of the
+ spec's loader.
+* Also see [cleaner_reload_support]_.
+
+The module spec `Factory Functions`_ could be classmethods on
+ModuleSpec. However that would expose them on *all* modules via
+``__spec__``, which has the potential to unnecessarily confuse
+non-advanced Python users. The factory functions have a specific use
+case, to support finder authors. See `ModuleSpec Users`_.
+
+Likewise, several other methods could be added to ModuleSpec that expose
+the specific uses of module specs by the import machinery:
+
+* create() - a wrapper around Loader.create_module().
+* exec(module) - a wrapper around Loader.exec_module().
+* load() - an analogue to the deprecated Loader.load_module().
+
+As with the factory functions, exposing these methods via
+module.__spec__ is less than desireable. They would end up being an
+attractive nuisance, even if only exposed as "private" attributes (as
+they were in previous versions of this PEP). If someone finds a need
+for these methods later, we can expose the via an appropriate API
+(separate from ModuleSpec) at that point, perhaps relative to PEP 406
+(import engine).
+
+Conceivably, the load() method could optionally take a list of
+modules with which to interact instead of sys.modules. Also, load()
+could be leveraged to implement multi-version imports. Both are
+interesting ideas, but definitely outside the scope of this proposal.
+
+Others left out:
+
+* Add ModuleSpec.submodules (RO-property) - returns possible submodules
+ relative to the spec.
+* Add ModuleSpec.loaded (RO-property) - the module in sys.module, if
+ any.
+* Add ModuleSpec.data - a descriptor that wraps the data API of the
+ spec's loader.
+* Also see [cleaner_reload_support]_.
+
+
References
==========
-.. [ref_files_pep] http://mail.python.org/pipermail/import-sig/2013-August/000658.html
+.. [ref_files_pep]
+ http://mail.python.org/pipermail/import-sig/2013-August/000658.html
.. [import_system_docs] http://docs.python.org/3/reference/import.html
-.. [cleaner_reload_support] https://mail.python.org/pipermail/import-sig/2013-September/000735.html
+.. [cleaner_reload_support]
+ https://mail.python.org/pipermail/import-sig/2013-September/000735.html
-.. [lazy_import_concerns] https://mail.python.org/pipermail/python-dev/2013-August/128129.html
+.. [lazy_import_concerns]
+ https://mail.python.org/pipermail/python-dev/2013-August/128129.html
+.. [reload-semantics-fix] http://bugs.python.org/issue19413
+
+.. [supports_reload_considered_harmful]
+ https://mail.python.org/pipermail/python-dev/2013-October/129971.html
+
+.. [exec_module_existing]
+ https://mail.python.org/pipermail/python-dev/2013-October/129933.html
Copyright
=========
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list