[Import-sig] Requirements

Gordon McMillan gmcm@hypernet.com
Fri, 4 Feb 2000 16:29:45 -0500


[me]
> > From the C API, we have PyImport_Import which does the 
> > same as (keyword) import. But PyImport_ImportModule and 
> > ...Ex are lower level. I assume that modulo some arg 
> > munging, these also will do the same as (keyword) import. 
> > Decent assumption?
[Guido]
> I suppose you mean they should do the same in the new design, because
> they would only be there for b/w compatibility?  

Yes.

> Right now they are
> designed to be different -- in particular PyImport_Import() calls
> __import__() calls PyImport_ImportModule[Ex]().
> 
> Do we want to keep the override-__import__ hook?

We need a builtin function (so you can use a runtime arg; and 
not be forced to exec). But there's not much sense in making 
it hookable, when the whole import system is a set of hooks.
 
> Who else uses PyImport_ImportModule[Ex]()?

In my experience, almost all extension writers use 
PyImport_ImportModule, not PyImport_Import. I think this is 
speed-freakism, not for functionality (which could only be to 
avoid hooks).
 
> > [Guido]
> > > - support for freeze functionality
> > 
> > Heh, heh. The current modulefinder works by (yet again) 
> > emulating the entire import process, but not letting the 
> > "imported" code leak out.

> > In imputil, it's the Importer base 
> > class that does the "leaking", not code in a (well-behaved) 
> > derived class. So that opens the possibility of replacing the 
> > Importer object in the derived class's bases with a 
> > PhonyImporter that doesn't leak. So modulefinder could use 
> > the derived class and wouldn't have to emulate. However, 
> > modulefinder would have to report more information - the 
> > importer that found the module, as well as the 
> > file/URL/whatever it found it in.
> 
> I'm afraid you've lost me here.  What does "leaking" refer to?

Letting the module into sys.modules or any real namespace. 
Just pointing out that a new modulefinder should be able to 
follow the hooks without excessive effort.
 
[zip files on or in sys.path...]
> > In the current version, items on sys.path can be directory 
> > names or importer instances. Obviously at startup, sys.path is 
> > nothing more that strings. Also (per other discussions), 
> > sys.path starts as a minimal boot path, and gets expanded 
> > from Python.
> > 
> > What is this mechanism?
> 
> Look at Modules/getpath.c and PC/getpathp.c.  Or are you asking about
> how the mechanism should be redesigned?

Yes. "Current version" meant "of imputil". Sorry.
 
> > Do we worry about:
> >  Network installations in heterogeneous environments?
> 
> Yes, by supporting sys.exec_prefix.  This has consequences for
> getpath.c, see there.  I think this support has lost its significance
> with the advent of fast disks, but I'm not going to fight millions of
> sysadmins stuck in the past, so we have to continue to support it.
> It's no big deal anyway.
> 
> >  Ditto in homogeneous environments?
> 
> How can you even tell?  Maybe I don't understand what you are talking
> about (and then my previous response also doesn't make sense?)

Terms: by "heterogeneous" I meant, eg, a Solaris server with 
Solaris, Windows and Linux clients. By "homogeneous" I 
meant clients (and probably server) are all binary compatible.
 
I *think* "homogeneous" is more-or-less solved when "multiple 
incompatible installations" is solved.

The added complexity of "heterogeneous" being the plat_xxx 
libraries (and what package authors have to do), which 
appears to be getting deprecated(?).

> >  Multiple incompatible installations?
> 
> Emphatically yes.  A Python binary should be able to find out where
> the rest of its installation is.  This is a platform specific problem
> (hence getpath.c and PC/getpathp.c).

Um, yes and no (to it being platform specific). Yes in that you 
can't follow symlinks on Windows, or easily get the absolute 
path name of the executable in some *nixen. No, in that I feel 
strongly (modulo some of the COM stuff below) that the 
psuedo-code should be the same - just think of distutils and 
package authors!
 
> Note that on Windows there's the added problem of Mark Hammond's COM
> support.  COM services implemented by Python can be started on the fly
> without starting python.exe, e.g. by embedding such a COM object in a
> Word document.  The consequence of this (I've been told) is that the
> python15.dll file must live in the system directory (\WinNT, \Windows,
> etc.).  This means that its path is useless to find the rest of the
> installation, and that's why we're using the registry.
> 
> I don't know if all of this is still true; I would think that if a COM
> support DLL lives somewhere else, the registry could point to it?  But
> who am I to argue with Microsoft.

I *think* this problem has been solved, and the registry can 
point wherever it wants, but I'm not the expert. If this stuff is 
still needed, perhaps it could be fallback: "Oops, I can't figure 
out PYTHONPATH, so I'll look in the registry". I'll forward this 
question to Mark.
 
> Anyway I wouldn't mind if this was somehow solved differently; 

Amen.

> > Should the syntax of .pth files be expanded to allow specifying 
> > importer instances? Or do we use sitecustomize?
> 
> Do you really think that will be used?  There would seem to be a
> chicken/egg problem.

Categorizing it doesn't solve it ;-). OK, we don't need a 
concrete solution now, but is this a reasonable approach?

1) Py_Initialize calls getpath.c
2) getpath.c returns a directory (or very short list thereof).
3) Fredrik drives his truck through (me too)
4) A frozen-in exceptions.py gets imported
5) sys.path gets expanded by looking for (something | some 
things) in the existing sys.path and ( executing | reading ) 
them.

Maybe 3 & 4 are swapped?

Maybe some of this is written in Python and frozen in?

The, err, obsession here being to make it (1) highly 
customizable AND (2) generally idiot-resistant. A few simple 
controls under a bright red hatch cover that says "Warning - 
touching this stuff will void your warranty".

- Gordon