[Python-Dev] imputil and modulefinder replacements

[Greg Ward]

On 05 October 2001, M.-A. Lemburg said:
> Huh ? The builtin mechanism in Python works just fine for packages.
[...]
> > A first-class solution would have these features IMHO:
> > 1) Access to libraries in zip files is implemented in C (I volunteer).
> 
> Great !
> 
> > 2) All Python library modules can be imported by C code,
> >    either from directories or zip files.
> 
> Right.
> 
> > 3) Package import is not implemented in C, it is implemented
> >    by a Python module which is part of the standard library.
> 
> Why ? I don't see a need for that and it would slow down 
> package imports which are becoming increasingly popular as the
> Python code base grows.

For the record, I agree with you on all points: there's no need to
rewrite package import in Python, but it would be nice to have "import
from archive" available in the core.  Quite possibly adding this feature
would silence 80% of the desire for genreic import hooks.

The only import hook I'm familiar with is Quixote, where we have defined
a Python dialect called PTL (Python Template Language, used to embed
HTML [or other text] in Python).  Install Quixote's import hook, and you
can import .ptl files just like .py files.  It's very handy.  Out of
curiosity, does anyone know of any other import hooks like this out
there -- ie. import something that is not strictly Python.  (It sounds
like most import hooks/hacks deal with the location of the .py files to
import.  Quixote doesn't touch that, but it adds the ability to import
not-quite-Python source files.)

> One question: should these ZIP-archives filenames be placed in 
> sys.path or should Python scan for ZIP-archives within the
> dirs on sys.path ?

I think the archive file should be listed in sys.path.

Note that my experience of this in Java was largely negative, because
Java doesn't have a standard way of putting .class files in the
filesystem (AFAIK) -- so everything has to be in a .jar file, and those
.jar files can be anywhere you please.  So you end up with a mile-long
CLASSPATH that's very fragile and forever needing fixing.  As long as
most Python modules are accessed the ordinary way (files in a
directory), then Python won't have a problem.  But if somebody makes a
Python installation with "stdlib.zip", "distutils.zip",
"mxDateTime.zip", etc. etc., then the poor users will be in the same
boat as Java users.

The main concerns I have with scanning sys.path directories for ZIP
files are performance and transparence.  (It ain't necessarily obvious
where a particular module will come from if sys.path can be searched in
two ways.)

        Greg
-- 
Greg Ward - Unix weenie                                 gward@python.net
http://starship.python.net/~gward/
Pointers are Arrays; Code is Data; Time is Money