[Python-Dev] zipimport.c broken with implicit namespace packages

[mike.romberg at comcast.net]

-- 
>>>>> "Brett" == Brett Cannon <brett at python.org> writes:

    > I opened
    > https://bugs.python.org/issue25711 to specifically try to
    > fix this issue once and for all and along the way modernize
    > zipimport by rewriting it from scratch to be more
    > maintainable

  Every time I read about impementing a custom loader:

https://docs.python.org/3/library/importlib.html

  I've wondered why python does not have some sort of virtual
filesystem layer to deal with locating modules/packages/support
files.   Virtual file systems seem like a good way to store data on a
wide range of storage devices.

  A VFSLoader object would interface with importlib and deal with:

  - implementing a finder and loader

  - Determine the actual type of file to load (.py, .pyc, .pyo,
    __pycache__, etc).

  - Do all of it's work by calling virtual functions such as:
     * listdir(path)
     * read(path)
     * stat(path)  # for things like mtime, size, etc
     * write(path, data)  # not all VFS implement this

  Then things like a ziploader would just inherit from VFSLoader
implement the straight forward methods and everything should "just
work" :).  I see no reason why every type of loader (real filesystem,
http, ssh, sql database, etc) would not do this as well.  Leave all
the details such as implicit namespace packages, presence of
__init__.py[oc] files, .pth files, etc in one single
location and put the details on how to interact with the actual
storage device in leaf classes which don't know or care about the high
level details.  They would not even know they are loading python
modules.  It is just blobs of data to them.

  I may try my hand at creating a prototype of this for just the
zipimporter and see how it goes.

  
    > At this point the best option might be, Mike, if you do a
    > code review for https://bugs.python.org/issue17633, even if
    > it is simply a LGTM. I will then personally make sure the
    > approved patch gets checked in for Python 3.6 in case the
    > rewrite of zipimport misses the release.

  Cool.  I'll see what I can do.  I was having a bit of trouble with
the register/login part of the bug tracker.  Which is why I came
here.  I'll battle with it one more time and see if I can get it to
log me in.

  The patch should be fairly simple.  In a nutshell it just does a:

  path.replace('.', '/') + '/' in two locations.  One where it checks
for the path being a directory entry in the zip file and the second to
return an implicit namespace path (instead of not found) if it is a
match.   I'll check the patch on the tracker and see if it still works
with 3.5.1.  If not I'll attach mine.

Mike