[Python-Dev] imputil and modulefinder replacements

[M.-A. Lemburg]

"James C. Ahlstrom" wrote:
> 
> Thomas Heller wrote:
> >
> > From: "Gordon McMillan" <gmcm@hypernet.com>
> > > I would like to propose these (or something quite like them) as
> > > replacements for the official versions. The code is quite similar
> > > (in fact, the modulefinder code could have been written by
> > > subclassing the imputil stuff, but I wrote them the other way
> > > 'round).
> > >
> > It seems noone cares about this. imputil is in the distribution,
> > but is it really 'official'?
> 
> I care about it.  I am using imputil to implement "jar" files,
> zip files of Python libraries.  Greg worked hard to get imputil
> into the distribution, and I appreciate his effort.  But I always
> wondered if imputil was 'official'.
> 
> Users on c.l.p frequently want to ship a stand-alone
> Python.  The current answer to this is freeze, but this requires a
> C compiler and some work.  A standard jar file implementation would
> satisfy this need.  What I am thinking of is documentation which
> says something like: "Zip up the Python libraries into pylibs.zip
> and put that file in the same directory as python[.exe] and
> everything Just Works."
> 
> My interest in imputil and ImportManager is to implement jar files,
> and neither does this well.  The problem is always how to get the
> SpecialImporter.py imported first, and how to hack the imports that
> the SpecialImporter.py may need to import.  In iu4.py we see the use
> of posix.stat to replace the os module, and the import of only
> built-in modules sys, imp, and marshal.  The same is true of imputil.py.
> 
> I admit I don't really understand package imports.  But I feel they
> are an unsolved problem, because package authors always seem to
> write custom ones.

Huh ? The builtin mechanism in Python works just fine for packages.
I think that custom importers are not very common these days. Still,
I do agree that the current implementation is a gross hack and should
in the long run be replaced by something more OO-style.

For the short-term however, I have a feeling that there are not
that many features actually being requested out there -- the most
wanted one is certainly the ability to import modules from some form
of archive, so perhaps we should start thinking about providing
this kind of support in the core ?!

> A first-class solution would have these features IMHO:
> 1) Access to libraries in zip files is implemented in C (I volunteer).

Great !

> 2) All Python library modules can be imported by C code,
>    either from directories or zip files.

Right.

> 3) Package import is not implemented in C, it is implemented
>    by a Python module which is part of the standard library.

Why ? I don't see a need for that and it would slow down 
package imports which are becoming increasingly popular as the
Python code base grows.

> 4) If an import of A.B is attempted, and a function
>    A.MyCustomImporter() exists, then that is used instead of
>    the default.

Better postpone this one...

> 5) More or better ideas from you all.

One question: should these ZIP-archives filenames be placed in 
sys.path or should Python scan for ZIP-archives within the
dirs on sys.path ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/