[Distutils] extensions in packages

[Greg Stein]

M.-A. Lemburg wrote:
> 
> Greg Stein wrote:
> >...
> > IMO, the interpreter core should perform as little searching as
> > possible. Basically, it should only contain bootstrap stuff. It should
> > look for a standard importing module and load that. After it is loaded,
> > the import mechanism should defer to Python for all future imports. (the
> > cost of running Python code is minimal against the I/O used by the
> > import)
> >
> > IMO #2, the standard importing module should operate along the lines of
> > imputil.py.
> 
> You mean moving the whole import mechanism away from C and into
> Python ? Have you tried such an approach with your imputil.py ?

Yes and yes.

Using Python's import hook effectively means that you completely take
over Python's import mechanism (one of its failings, IMO). imputil.py is
designed to provide for iterating through a list of importers, looking
for one that works.

In any case... yes, I've use imputil to the exclusion of Python's import
logic. You still need imp.new_module() and imp.get_magic(). But that
does implies that you can axe a lot of stuff outside of that. My tests
don't have loading of dynamic libraries, so you would still need an imp
function to load that (but strip the *searching* for the module).

> I wonder whether all things done in import.c can be coded in Python,
> esp. the exotic things like the Windows registry stuff and the
> Mac fork munging seem to be C only (at least as long as there are
> no core Python APIs for these C calls).

win32api provides Registry access, so you just have to bootstrap that. I
haven't tried to remove a lot of Python's logic, so I can't say what can
actually be tossed, kept around, or just restructured a bit. IMO, the
best thing to do is to expose a few minimal functions and defer to
Python.

> And just curious: why did Guido recode ni.py in C if he could have
> used ni.py in your proposed way instead ?

For two reasons that I can think of:

1) people had to manually import "ni"
2) it took over the import hook which effectively prevents further use
of it (or if somebody *did* use it, then they would wipe out ni's
functionality; again, this is why I dislike the current hook approach
and like a list-based approach which is possible via imputil)

And rather than respond to Fred's note in a separate thread, I'll tie it
in here:

Frankly: Fred is off-based on the need to "recode in C for efficiency".
That is a bogus argument. The cost is I/O operations, not the
interpreter overhead. You will gain no real benefit by moving the import
mechanism to C. C is *only* required to access the operating system in
ways that are not already available in the core, or which you cannot
effectively bootstrap.

Python should strip away all of its C-based code for packages and for
locating modules. That should all move to Python. All that should remain
in C is the necessary functions for importing dynamic modules.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/