On Wed, May 23, 2012 at 10:40 AM, Eric V. Smith eric@trueblade.com wrote:
On 5/22/2012 2:37 PM, Guido van Rossum wrote:
Okay, I've been convinced that keeping the dynamic path feature is a good idea. I am really looking forward to seeing the rationale added to the PEP -- that's pretty much the last thing on my list that made me hesitate. I'll leave the details of exactly how the parent path is referenced up to the implementation team (several good points were made), as long as the restriction that sys.path must be modified in place is lifted.
I've updated the PEP. Let me know how it looks.
I have not updated the implementation yet. I'm not exactly sure how I'm going to convert from a path list of unknown origin to ('sys', 'path') or ('foo', '__path__'). I'll look at it later tonight to see if it's possible. I'm hoping it doesn't require major surgery to importlib._bootstrap.
If you wanted to do this without changing the sys.meta_path hook API, you'd have to pass an object to find_module() that did the dynamic lookup of the value in obj.__iter__. Something like:
class _LazyPath: def __init__(self, modname, attribute): self.modname = modname self.attribute = attribute def __iter__(self): return iter(getattr(sys.module[self.modname], self.attribute))
A potentially cleaner alternative to consider is tweaking the find_loader API spec so that it gets used at the meta path level as well as at the path hooks level and is handed a *callable* that dynamically retrieves the path rather than a direct reference to the path itself.
The full signature of find_loader would then become:
def find_loader(fullname, get_path=None): # fullname as for find_module # When get_path is None, it means the finder is being called as a path hook and # should use the specific path entry passed to __init__ # In this case, namespace package portions are returned as (None, portions) # Otherwise, the finder is being called as a meta_path hook and get_path() will return the relevant path # Any namespace packages are then returned as (loader, portions)
There are two major consequences of this latter approach: - the PEP 302 find_module API would now be a purely legacy interface for both the meta_path and path_hooks, used only if find_loader is not defined - it becomes trivial to tell whether a particular name references a package or not *without* needing to load it first: find_loader() returns a non-empty iterable for the list of portions
That second consequence is rather appealing: it means you'd be able to implement an almost complete walk of a package hierarchy *without* having to import anything (although you would miss old-style namespace packages and any other packages that alter their own __path__ in __init__, so you may still want to load packages to make sure you found everything. You could definitively answer the "is this a package or not?" question without running any code, though).
The first consequence is also appealing, since the find_module() name is more than a little misleading. The "find_module" name strongly suggests that the method is expected to return a module object, and that's just wrong - you actually find a loader, then you use that to load the module.
Cheers, Nick.