[Python-Dev] Draft PEP: "Simplified Package Layout and Partitioning"

[P.J. Eby]

At 02:24 AM 7/20/2011 -0700, Glenn Linderman wrote:
>When I read about creating __path__ from sys.path, I immediately 
>thought of the issue of programs that extend sys.path, and the above 
>is the "workaround" for such programs.  but it requires such 
>programs to do work, and there are a lot of such programs (I, a 
>relative newbie, have had to write some).  As it turns out, I can't 
>think of a situation where I have extended sys.path that would 
>result in a problem for fancy namespace packages, because so far 
>I've only written modules, not packages, and only modules are on the 
>paths that I add to sys.path.  But that does not make for a general solution.

Most programs extend sys.path in order to import things.  If those 
things aren't yet imported, they don't have a __path__ yet, and so 
don't need to be fixed.  Only programs that modify sys.path *after* 
importing something that has a dynamic __path__ would need to do 
anything about that.


>Is there some way to create a new __path__ that would reflect the 
>fact that it has been dynamically created, rather than set from 
>__init__.py, and then when it is referenced, calculate (and cache?) 
>a new value of __path__ to actually search?

That's what extend_virtual_paths() is for.  It updates the __path__ 
of all currently-imported virtual packages.  Where before you wrote:

      sys.path.append('foo')

You would now write:

      sys.path.append('foo')
      pkgutil.extend_virtual_paths('foo')

...assuming you have virtual packages you've already imported.  If 
you don't, there's no reason to call extend_virtual_paths().  But it 
doesn't hurt anything if you call it unnecessarily, because it uses 
sys.virtual_packages to find out what to update, and if you haven't 
imported any virtual packages, there's nothing to update and the call 
will be a quick no-op.