On 05/21/2012 07:25 PM, Nick Coghlan wrote:
As a simple example to back up PJE's explanation, consider:
- encodings becomes a namespace package
- It sometimes gets imported during interpreter startup to initialise
the standard io streams 3. An application modifies sys.path after startup and wants to contribute additional encodings
Searching the entire parent path for new portions on every import would be needlessly slow.
Not recognising new portions would be needlessly confusing for users. In our simple case above, the application would fail if the io initialisation accessed the encodings package, but work if it did not (e.g. when all streams are utf-8).
PEP 420 splits the difference via an automatically invalidated cache: when you iterate over a namespace package __path__ object, it rescans the parent path for new portions *if and only if* the contents of the parent path have changed since the previous scan.
That seems like a pretty convincing example to me.
Personally I'm +1 on putting dynamic computation into the PEP, at least for top-level namespace packages, and probably for all namespace packages.
The code is not very large or complicated, and with the proposed removal of the restriction that sys.path cannot be replaced, I think it behaves well.
But Guido can decide against it without hurting my feelings.
Eric.
P.S.: Here's the current code in the pep-420 branch. This code still has the restriction that sys.path (or parent_path in general) can't be replaced. I'll fix that if we decide to keep the feature.
class _NamespacePath: def __init__(self, name, path, parent_path, path_finder): self._name = name self._path = path self._parent_path = parent_path self._last_parent_path = tuple(parent_path) self._path_finder = path_finder
def _recalculate(self): # If _parent_path has changed, recalculate _path parent_path = tuple(self._parent_path) # Make a copy if parent_path != self._last_parent_path: loader, new_path = self._path_finder(self._name, parent_path) # Note that no changes are made if a loader is returned, but we # do remember the new parent path if loader is None: self._path = new_path self._last_parent_path = parent_path # Save the copy return self._path
def __iter__(self): return iter(self._recalculate())
def __len__(self): return len(self._recalculate())
def __repr__(self): return "_NamespacePath" + repr((self._path, self._parent_path))
def __contains__(self, item): return item in self._recalculate()