[Distutils] extensions in packages

[M.-A. Lemburg]

Greg Stein wrote:
> 
> M.-A. Lemburg wrote:
> >
> > Greg Stein wrote:
> > >...
> > > IMO, the interpreter core should perform as little searching as
> > > possible. Basically, it should only contain bootstrap stuff. It should
> > > look for a standard importing module and load that. After it is loaded,
> > > the import mechanism should defer to Python for all future imports. (the
> > > cost of running Python code is minimal against the I/O used by the
> > > import)
> > >
> > > IMO #2, the standard importing module should operate along the lines of
> > > imputil.py.
> >
> > You mean moving the whole import mechanism away from C and into
> > Python ? Have you tried such an approach with your imputil.py ?
> 
> Yes and yes.
> 
> Using Python's import hook effectively means that you completely take
> over Python's import mechanism (one of its failings, IMO). imputil.py is
> designed to provide for iterating through a list of importers, looking
> for one that works.
>
> In any case... yes, I've use imputil to the exclusion of Python's import
> logic. You still need imp.new_module() and imp.get_magic(). But that
> does implies that you can axe a lot of stuff outside of that. My tests
> don't have loading of dynamic libraries, so you would still need an imp
> function to load that (but strip the *searching* for the module).

So all that's needed is some DLL loader magic in C, some Win32
Registry APIs and the Mac fork stuff. If imp were extended to
provide those APIs instead of using them itself and then moving
all the other code to Python we should arrive at a solution
that would also cover things like internet import, import via
pipes, import from databases, etc.

> > And just curious: why did Guido recode ni.py in C if he could have
> > used ni.py in your proposed way instead ?
> 
> For two reasons that I can think of:
> 
> 1) people had to manually import "ni"

Well, I guess ni could have been imported by Python at startup
-- just like exceptions is right now. The problem here: what if
it doesn't find the import logic module ? Plus, how does it
do import ;) ?

> 2) it took over the import hook which effectively prevents further use
> of it (or if somebody *did* use it, then they would wipe out ni's
> functionality; again, this is why I dislike the current hook approach
> and like a list-based approach which is possible via imputil)

Right. BTW: that's a general problem with all kinds of hooks,
e.g. the system exit hook is another problem area. A generic
solution to this wouldn't be a bad idea either.

Here is one way to do this for the sys.exitfunc hook that I'm
currently using:

""" Central Registry for sys.exitfunc()-type functions.
"""
__version__ = "0.1"

import sys,traceback

class ExitFunctionDispatcher:

    """ Singleton that manages exit functions. These function will be
        called upon system exit in reverse order of their registering.
    """
    def __init__(self):

        """ Install the dispatcher as sys.exitfunc()
        """
        self.exitfunc_list = []
        if hasattr(sys,'exitfunc'):
            self.old_exitfunc = sys.exitfunc
        else:
            self.old_exitfunc = None
        sys.exitfunc = self.exitfunc

    def exitfunc(self,

                 write=sys.stderr.write,print_exc=traceback.print_exc,
                 stderr=sys.stderr):

        """ This is the exitfunc that we install to dispatch the
            processing to the registered other functions
        """
        for f in self.exitfunc_list:
            try:
                f()
            except:
                write('Error while executing Exitfunction %s:\n' % f.__name__)
                print_exc(10,stderr)
        # Now that we're finished, call the previously installed exitfunc()
        if self.old_exitfunc:
            self.old_exitfunc()

    def register(self,f,position=0):

        """ Register f as exit function. These functions must not take
            parameters.
            - position = 0: register the function at the beginning of the
              list; these functions get called before the functions already
              in the list (default)
            - position = -1: register the function at the end of the list;
              the function will get called after all other functions
        """
        if position < 0: 
            position = position + len(self.exitfunc_list) + 1
        self.exitfunc_list.insert(position,f)

    def deregister(self,f):

        """ Remove the function f from the exitfunc list; if it is not
            found, the error is silently ignored.
        """
        try:
            self.exitfunc_list.remove(f)
        except:
            pass

# Create the singleton
ExitFunctions = ExitFunctionDispatcher()

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                   213 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/