[Import-sig] Requirements
Gordon McMillan
gmcm@hypernet.com
Fri, 4 Feb 2000 16:29:45 -0500
[me]
> > From the C API, we have PyImport_Import which does the
> > same as (keyword) import. But PyImport_ImportModule and
> > ...Ex are lower level. I assume that modulo some arg
> > munging, these also will do the same as (keyword) import.
> > Decent assumption?
[Guido]
> I suppose you mean they should do the same in the new design, because
> they would only be there for b/w compatibility?
Yes.
> Right now they are
> designed to be different -- in particular PyImport_Import() calls
> __import__() calls PyImport_ImportModule[Ex]().
>
> Do we want to keep the override-__import__ hook?
We need a builtin function (so you can use a runtime arg; and
not be forced to exec). But there's not much sense in making
it hookable, when the whole import system is a set of hooks.
> Who else uses PyImport_ImportModule[Ex]()?
In my experience, almost all extension writers use
PyImport_ImportModule, not PyImport_Import. I think this is
speed-freakism, not for functionality (which could only be to
avoid hooks).
> > [Guido]
> > > - support for freeze functionality
> >
> > Heh, heh. The current modulefinder works by (yet again)
> > emulating the entire import process, but not letting the
> > "imported" code leak out.
> > In imputil, it's the Importer base
> > class that does the "leaking", not code in a (well-behaved)
> > derived class. So that opens the possibility of replacing the
> > Importer object in the derived class's bases with a
> > PhonyImporter that doesn't leak. So modulefinder could use
> > the derived class and wouldn't have to emulate. However,
> > modulefinder would have to report more information - the
> > importer that found the module, as well as the
> > file/URL/whatever it found it in.
>
> I'm afraid you've lost me here. What does "leaking" refer to?
Letting the module into sys.modules or any real namespace.
Just pointing out that a new modulefinder should be able to
follow the hooks without excessive effort.
[zip files on or in sys.path...]
> > In the current version, items on sys.path can be directory
> > names or importer instances. Obviously at startup, sys.path is
> > nothing more that strings. Also (per other discussions),
> > sys.path starts as a minimal boot path, and gets expanded
> > from Python.
> >
> > What is this mechanism?
>
> Look at Modules/getpath.c and PC/getpathp.c. Or are you asking about
> how the mechanism should be redesigned?
Yes. "Current version" meant "of imputil". Sorry.
> > Do we worry about:
> > Network installations in heterogeneous environments?
>
> Yes, by supporting sys.exec_prefix. This has consequences for
> getpath.c, see there. I think this support has lost its significance
> with the advent of fast disks, but I'm not going to fight millions of
> sysadmins stuck in the past, so we have to continue to support it.
> It's no big deal anyway.
>
> > Ditto in homogeneous environments?
>
> How can you even tell? Maybe I don't understand what you are talking
> about (and then my previous response also doesn't make sense?)
Terms: by "heterogeneous" I meant, eg, a Solaris server with
Solaris, Windows and Linux clients. By "homogeneous" I
meant clients (and probably server) are all binary compatible.
I *think* "homogeneous" is more-or-less solved when "multiple
incompatible installations" is solved.
The added complexity of "heterogeneous" being the plat_xxx
libraries (and what package authors have to do), which
appears to be getting deprecated(?).
> > Multiple incompatible installations?
>
> Emphatically yes. A Python binary should be able to find out where
> the rest of its installation is. This is a platform specific problem
> (hence getpath.c and PC/getpathp.c).
Um, yes and no (to it being platform specific). Yes in that you
can't follow symlinks on Windows, or easily get the absolute
path name of the executable in some *nixen. No, in that I feel
strongly (modulo some of the COM stuff below) that the
psuedo-code should be the same - just think of distutils and
package authors!
> Note that on Windows there's the added problem of Mark Hammond's COM
> support. COM services implemented by Python can be started on the fly
> without starting python.exe, e.g. by embedding such a COM object in a
> Word document. The consequence of this (I've been told) is that the
> python15.dll file must live in the system directory (\WinNT, \Windows,
> etc.). This means that its path is useless to find the rest of the
> installation, and that's why we're using the registry.
>
> I don't know if all of this is still true; I would think that if a COM
> support DLL lives somewhere else, the registry could point to it? But
> who am I to argue with Microsoft.
I *think* this problem has been solved, and the registry can
point wherever it wants, but I'm not the expert. If this stuff is
still needed, perhaps it could be fallback: "Oops, I can't figure
out PYTHONPATH, so I'll look in the registry". I'll forward this
question to Mark.
> Anyway I wouldn't mind if this was somehow solved differently;
Amen.
> > Should the syntax of .pth files be expanded to allow specifying
> > importer instances? Or do we use sitecustomize?
>
> Do you really think that will be used? There would seem to be a
> chicken/egg problem.
Categorizing it doesn't solve it ;-). OK, we don't need a
concrete solution now, but is this a reasonable approach?
1) Py_Initialize calls getpath.c
2) getpath.c returns a directory (or very short list thereof).
3) Fredrik drives his truck through (me too)
4) A frozen-in exceptions.py gets imported
5) sys.path gets expanded by looking for (something | some
things) in the existing sys.path and ( executing | reading )
them.
Maybe 3 & 4 are swapped?
Maybe some of this is written in Python and frozen in?
The, err, obsession here being to make it (1) highly
customizable AND (2) generally idiot-resistant. A few simple
controls under a bright red hatch cover that says "Warning -
touching this stuff will void your warranty".
- Gordon