From thomas.heller@ion-tof.com  Tue Jan 30 19:23:17 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Tue, 30 Jan 2001 20:23:17 +0100
Subject: [Import-sig] Imputil
Message-ID: <00c301c08af2$19e85870$e000a8c0@thomasnotebook>

I have the impression that we cannot currently
import cPickle (for example) by imputil:

cPickle uses PyImport_ImportModule(name) instead of
PyImport_Import(name) which would then in turn use the
import hook function.
cPickle imports copy_reg, string, types.

Is this observation correct?

Regards,

Thomas



From mal@lemburg.com  Wed Jan 31 14:25:06 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 31 Jan 2001 15:25:06 +0100
Subject: [Import-sig] Imputil
References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook>
Message-ID: <3A782042.B845D04E@lemburg.com>

Thomas Heller wrote:
> 
> I have the impression that we cannot currently
> import cPickle (for example) by imputil:
> 
> cPickle uses PyImport_ImportModule(name) instead of
> PyImport_Import(name) which would then in turn use the
> import hook function.
> cPickle imports copy_reg, string, types.
> 
> Is this observation correct?

>From looking at the cPickle.c code: no. cPickle comes with its
own PyImport_Import() API which looks very similar to the
one in Python/import.c (probably there for historic reasons).
The cPickle implementation uses __import__ as well, so there
should be no problem.

The various uses of PyImport_ImportModule() in cPickle are only
used for standard modules -- I don't think these pose much of
a problem w/r to imputil and other import hooks, since these
are either available using the standard import mechanism or
as frozen modules compiled into the core.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/


From thomas.heller@ion-tof.com  Wed Jan 31 14:52:42 2001
From: thomas.heller@ion-tof.com (Thomas Heller)
Date: Wed, 31 Jan 2001 15:52:42 +0100
Subject: [Import-sig] Imputil
References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> <3A782042.B845D04E@lemburg.com>
Message-ID: <007d01c08b95$77d241f0$e000a8c0@thomasnotebook>

"M.-A. Lemburg" <mal@lemburg.com> wrote:
> Thomas Heller wrote:
> > 
> > I have the impression that we cannot currently
> > import cPickle (for example) by imputil:
> > 
> > cPickle uses PyImport_ImportModule(name) instead of
> > PyImport_Import(name) which would then in turn use the
> > import hook function.
> > cPickle imports copy_reg, string, types.
> > 
> > Is this observation correct?
> 
> >From looking at the cPickle.c code: no. cPickle comes with its
> own PyImport_Import() API which looks very similar to the
> one in Python/import.c (probably there for historic reasons).
> The cPickle implementation uses __import__ as well, so there
> should be no problem.
> 
> The various uses of PyImport_ImportModule() in cPickle are only
> used for standard modules -- I don't think these pose much of
> a problem w/r to imputil and other import hooks, since these
> are either available using the standard import mechanism or
> as frozen modules compiled into the core.

No, these are exactly the problem (maybe only my problem?).
If the standard import mechanism is NOT available (think of
Gordon's installer or my py2exe project) because everything is loaded
from an archive, importing cPickle will fail if you do:

import cPickle
import copy_reg, string, types

but succeed if you do:

import copy_reg, string, types
import cPickle

because in the latter case copy_reg, string, types are
already in sys.modules when 'import cPickle' is executed.

Thomas



From mal@lemburg.com  Wed Jan 31 15:05:39 2001
From: mal@lemburg.com (M.-A. Lemburg)
Date: Wed, 31 Jan 2001 16:05:39 +0100
Subject: [Import-sig] Imputil
References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> <3A782042.B845D04E@lemburg.com> <007d01c08b95$77d241f0$e000a8c0@thomasnotebook>
Message-ID: <3A7829C3.488DE50A@lemburg.com>

Thomas Heller wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com> wrote:
> > Thomas Heller wrote:
> > >
> > > I have the impression that we cannot currently
> > > import cPickle (for example) by imputil:
> > >
> > > cPickle uses PyImport_ImportModule(name) instead of
> > > PyImport_Import(name) which would then in turn use the
> > > import hook function.
> > > cPickle imports copy_reg, string, types.
> > >
> > > Is this observation correct?
> >
> > >From looking at the cPickle.c code: no. cPickle comes with its
> > own PyImport_Import() API which looks very similar to the
> > one in Python/import.c (probably there for historic reasons).
> > The cPickle implementation uses __import__ as well, so there
> > should be no problem.
> >
> > The various uses of PyImport_ImportModule() in cPickle are only
> > used for standard modules -- I don't think these pose much of
> > a problem w/r to imputil and other import hooks, since these
> > are either available using the standard import mechanism or
> > as frozen modules compiled into the core.
> 
> No, these are exactly the problem (maybe only my problem?).
> If the standard import mechanism is NOT available (think of
> Gordon's installer or my py2exe project) because everything is loaded
> from an archive, importing cPickle will fail if you do:
> 
> import cPickle
> import copy_reg, string, types
> 
> but succeed if you do:
> 
> import copy_reg, string, types
> import cPickle
> 
> because in the latter case copy_reg, string, types are
> already in sys.modules when 'import cPickle' is executed.

Perhaps you should add some "boot" code to py2exe then or
freeze the standard lib into the interpreter itself ?!
(mxCGIPython works this way.)

Changing cPickle.c doesn't help here: the PyImport_ImportModule()
API would have to be made __import__ aware to fix this problem
because a lot of Python code including the core itself uses
this more direct import API.

OTOH, it may sometimes be necessary to explicitly use the
direct and builtin import mechanism (e.g. during Python
startup)... so perhaps the playing with PyImport_ImportModule()
isn't such a good idea after all.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Company:                                        http://www.egenix.com/
Consulting:                                    http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/