At 03:22 PM 7/20/2011 -0600, Eric Snow wrote:

On Wed, Jul 20, 2011 at 2:44 PM, P.J. Eby <pje@telecommunity.com> wrote:

...

So, yeah, actually, that's looking pretty sweet. Basically, we

just have to

...

throw a virtual_package_paths dict into the sys module, and do the above along with the get_virtual_path() function and add get_subpath() to the importer objects, in order to get the PEP's core functionality working.

Exactly. That's part of why the importlib approach is so appealing to me.

Actually, it turns out I was a little too optimistic -- the sketch I gave doesn't work right for anything but top-level virtual packages, because I didn't take into account the part where get_virtual_path() needs a parent path. Fixing *that* error then leads to a really nasty bit of mutual recursion in which the parent module imports are attempted over and over again in something like O(N**2), I think. In order to get rid of that, _gcd_import would have to grow some internal memoization so it doesn't retry the same imports repeatedly. Ironically enough, this is because _gcd_import() is recursive, and thus attempts the imports in the opposite order (sort of) than import.c does, which means that you can't get hold of the parent's __path__ without recursing (again). :-( And trying to work around that with memoization, led me to the realization that you actually can't implement PEP 402 using that type of recursion. That is, to implement the spec correctly, _gcd_import is going to have to be refactored to iterate left-to-right over module name parts, rather than recursing right-to-left. That's because PEP 402 only allows for processing a virtual path if a module is not found, *not* if a module is found but can't be loaded. But, with importlib currently being recursive, it only knows that a parent import failed via ImportError, not whether that error arose from failing to find the module, or failing to load the module! So, the core part of the _gcd_import() function will need to be rewritten to iterate instead of recursing. (Still, it's probably not going to be *terribly* difficult. I'll take a look at doing a sketch of that next, but if I do one I'll send it to Import-SIG instead of here; it's not a detail that matters to the general PEP discussion.)