[Python-Dev] [Python-checkins] cpython: Issue #15502: Bring the importlib.PathFinder docs and docstring more in line

[Barry Warsaw]

On Aug 02, 2012, at 03:04 PM, nick.coghlan wrote:

>http://hg.python.org/cpython/rev/1f8351cf00f3
>changeset:   78382:1f8351cf00f3
>user:        Nick Coghlan <ncoghlan at gmail.com>
>date:        Thu Aug 02 23:03:58 2012 +1000
>summary:
>  Issue #15502: Bring the importlib.PathFinder docs and docstring more in line with the new import system documentation, and fix various parts of the new docs that weren't quite right given PEP 420 or were otherwise a bit misleading. Also note the key terminology problem still being discussed in the issue
>
>files:
>  Doc/library/importlib.rst   |    15 +-
>  Doc/reference/import.rst    |   221 +-
>  Lib/importlib/_bootstrap.py |     8 +-
>  Python/importlib.h          |  2583 +++++++++++-----------
>  4 files changed, 1449 insertions(+), 1378 deletions(-)
>
>
>diff --git a/Doc/reference/import.rst b/Doc/reference/import.rst
>--- a/Doc/reference/import.rst
>+++ b/Doc/reference/import.rst
>@@ -69,7 +75,7 @@
> 
> It's important to keep in mind that all packages are modules, but not all
> modules are packages.  Or put another way, packages are just a special kind of
>-module.  Specifically, any module that contains an ``__path__`` attribute is
>+module.  Specifically, any module that contains a ``__path__`` attribute is

I find this change hilarious!  Is it "an under-under path" or "a dunder path"?
Personally, I think word "dunder" should never be used unless it's followed by
the word "mifflin", but that might be a bit too regional.  Or maybe too old
skool (yeah, I know all the kids love the dunder).

I suppose I don't care that much, but it would be useful to have some
consistency, like whether we use American or British spellings.

>@@ -90,7 +96,7 @@
> packages are traditional packages as they existed in Python 3.2 and earlier.
> A regular package is typically implemented as a directory containing an
> ``__init__.py`` file.  When a regular package is imported, this
>-``__init__.py`` file is implicitly imported, and the objects it defines are
>+``__init__.py`` file is implicitly executed, and the objects it defines are

Perhaps "loaded" instead of either "imported" or "executed"?

>@@ -107,9 +113,9 @@
>         three/
>             __init__.py
> 
>-Importing ``parent.one`` will implicitly import ``parent/__init__.py`` and
>+Importing ``parent.one`` will implicitly execute ``parent/__init__.py`` and

Same.

> ``parent/one/__init__.py``.  Subsequent imports of ``parent.two`` or
>-``parent.three`` will import ``parent/two/__init__.py`` and
>+``parent.three`` will execute ``parent/two/__init__.py`` and

And here.

> ``parent/three/__init__.py`` respectively.
> 
> 
>@@ -128,6 +134,12 @@
> objects on the file system; they may be virtual modules that have no concrete
> representation.
> 
>+Namespace packages do not use an ordinary list for their ``__path__``
>+attribute. They instead use a custom iterable type which will automatically
>+perform a new search for package portions on the next import attempt within
>+that package if the path of their parent package (or :data:`sys.path` for a
>+top level package) changes.

Nice addition.  I can't help but suggest a slight rephrasing:

Namespace packages use a custom iterable type for their ``__path__``
attribute, so that changes in their parent package's path (or :data:`sys.path`
for a top level package) are handled automatically.

>@@ -172,14 +184,18 @@
> :exc:`ImportError` is raised.  If the module name is missing, Python will
> continue searching for the module.
> 
>-:data:`sys.modules` is writable.  Deleting a key will not destroy the
>-associated module, but it will invalidate the cache entry for the named
>-module, causing Python to search anew for the named module upon its next
>-import.  Beware though, because if you keep a reference to the module object,
>+:data:`sys.modules` is writable.  Deleting a key may not destroy the
>+associated module (as other modules may hold references to it),

s/as other modules may hold references to it/due to circular references/

>@@ -369,7 +404,7 @@
> 
> Here are the exact rules used:
> 
>- * If the module has an ``__loader__`` and that loader has a
>+ * If the module has a ``__loader__`` and that loader has a
>    :meth:`module_repr()` method, call it with a single argument, which is the
>    module object.  The value returned is used as the module's repr.
> 
>@@ -377,10 +412,10 @@
>    and discarded, and the calculation of the module's repr continues as if
>    :meth:`module_repr()` did not exist.
> 
>- * If the module has an ``__file__`` attribute, this is used as part of the
>+ * If the module has a ``__file__`` attribute, this is used as part of the
>    module's repr.
> 
>- * If the module has no ``__file__`` but does have an ``__loader__``, then the
>+ * If the module has no ``__file__`` but does have a ``__loader__``, then the
>    loader's repr is used as part of the module's repr.
> 
>  * Otherwise, just use the module's ``__name__`` in the repr.

C'mon Michael Scott, make it stop!  Do you remember the episode where Dwight
and Jim team up to...  Oh, I was in stitches!

>@@ -430,15 +467,20 @@
> path`, which contains a list of :term:`path entries <path entry>`.  Each path
> entry names a location to search for modules.
> 
>-Path entries may name file system locations, and by default the :term:`path
>-importer` knows how to provide traditional file system imports.  It implements
>-all the semantics for finding modules on the file system, handling special
>-file types such as Python source code (``.py`` files), Python byte code
>-(``.pyc`` and ``.pyo`` files) and shared libraries (e.g. ``.so`` files).
>+The path importer itself doesn't know how to import anything. Instead, it
>+traverses the individual path entries, associating each of them with a
>+path entry finder that knows how to handle that particular kind of path.
>+
>+The default set of path entry finders implement all the semantics for finding
>+modules on the file system, handling special file types such as Python source
>+code (``.py`` files), Python byte code (``.pyc`` and ``.pyo`` files) and
>+shared libraries (e.g. ``.so`` files). When supported by the :mod:`zipimport`
>+module in the standard library, the default path entry finders also handle
>+loading all of these file types (other than shared libraries) from zipfiles.

s/zipfiles/zip files/

> 
> Path entries need not be limited to file system locations.  They can refer to
>-the contents of zip files, URLs, database queries, or any other location that
>-can be specified as a string.
>+the URLs, database queries, or any other location that can be specified as a
>+string.

s/the URLs/URLs/

> 
> The :term:`path importer` provides additional hooks and protocols so that you
> can extend and customize the types of searchable path entries.  For example,
>@@ -534,29 +578,59 @@
> Path entry finder protocol
> --------------------------
> 
>-Path entry finders support the same :meth:`find_module()` method that meta
>-path finders support, however path entry finder's :meth:`find_module()`
>-methods are never called with a ``path`` argument.
>-
>-The :meth:`find_module()` method on path entry finders is deprecated though,
>-and instead path entry finders should implement the :meth:`find_loader()`
>-method.  If it exists on the path entry finder, :meth:`find_loader()` will
>-always be called instead of :meth:`find_module()`.
>+In order to support imports of modules and initialized packages and also to
>+contribute portions to namespace packages, path entry finders must implement
>+the :meth:`find_loader()` method.

This is an improvement, because it de-emphasizes the deprecated API.  Perhaps
add a footnote that describes the legacy API?

> :meth:`find_loader()` takes one argument, the fully qualified name of the
> module being imported.  :meth:`find_loader()` returns a 2-tuple where the
> first item is the loader and the second item is a namespace :term:`portion`.
> When the first item (i.e. the loader) is ``None``, this means that while the
>-path entry finder does not have a loader for the named module, it knows that
>-the :term:`path entry` contributes to a namespace portion for the named
>-module.  This will almost always be the case where Python is asked to import a
>-:term:`namespace package` that has no physical presence on the file system.
>-When a path entry finder returns ``None`` for the loader, the second item of
>-the 2-tuple return value must be a sequence, although it can be empty.
>+path entry finder does not have a loader for the named module, it knows that the
>+path entry contributes to a namespace portion for the named module.  This will
>+almost always be the case where Python is asked to import a namespace package
>+that has no physical presence on the file system.  When a path entry finder
>+returns ``None`` for the loader, the second item of the 2-tuple return value
>+must be a sequence, although it can be empty.
> 
> If :meth:`find_loader()` returns a non-``None`` loader value, the portion is
>-ignored and the loader is returned from the :term:`path importer`, terminating
>-the :term:`import path` search.
>+ignored and the loader is returned from the path importer, terminating the
>+search through the path entries.
>+
>+For backwards compatibility with other implementations of the import
>+protocol, many path entry finders also support the same,
>+traditional :meth:`find_module()` method that meta path finders support.
>+However path entry finder :meth:`find_module()` methods are never called
>+with a ``path`` argument (they are expected to record the appropriate
>+path information from the initial call to the path hook).
>+
>+The :meth:`find_module()` method on path entry finders is deprecated,
>+as it does not allow the path entry finder to contribute portions to
>+namespace packages. Instead path entry finders should implement the
>+:meth:`find_loader()` method as described above. If it exists on the path
>+entry finder, the import system will always call :meth:`find_loader()`
>+in preference to :meth:`find_module()`.

The above is what could be moved to the footnote.

Cheers,
-Barry