[Distutils] Freeze and new import architecture

John Skaller skaller@maxtal.com.au
Sun, 20 Dec 1998 04:17:11 +1000


At 18:15 17/12/98 -0800, Greg Stein wrote:
>> ...
>> sys.path not be restricted to path names. 
> sys.path has "strings", and
>> an associated map of "module finders".  Thus, a sys.path entry could
>> have a directory name (like now) or .zip file, URL, etc.
>
>I would much prefer to see the module finder instances in the sys.path.

        I agree. But that would be a compatibility problem?

>Sometimes, it is *very* difficult to map strings to module finders. 

        Yes. I think this is a serious weakness of the proposal.

>> 1. Finding the module in a specific namespace.
>> 2. Importing a module of a specific type, once it has been found.
>
>I think the separation is bogus. 

        I don't: I'd _like_ to add two things, as a client
of the system:

        1) Add a new type. For example, allow .c files to be loaded,
           by compiling them first.

        2) Add a new kind of namespace. For example, an FTP server.
           Or a hook to Trove. Or a private data structure I designed
           myself.

(1) has to do with what kinds of things are loaded, 
whereas (2) has to do with where they are.

Note that the finder must be able to _fetch_ the data to a place
that the loader can load it from. If a single place is enough,
then the two features are orthogonal, and thus each is separately
amenable to Object Oriented development.

        So .. it is desirable to build an abstraction in which
the functionality is separate.


>> Regarding 2: the finder currently returns a structure that enables the
>> correct
>> importer to be called later on. Importers that we have are for
>> builtin,
>> frozen, .py/.pyc/.pyo modules, various dll-importers all hiding behind
>> the
>> same interface, PYC-resource importers (mac-only) and PYD-resource
>> importers
>> (mac-only).
>
>Punt this. Just import the dumb thing in one shot.

        But you can't: a .dll file is imported by saying dlopen(),
whereas a .py file is imported by compiling it to a .pyc file which
is then imported. Etc.

        'One shot' implies a single function which is not extensible.

>Take the example of an HTTP-based import. Separating that into *two*
>transactions would be painful. It should be imported in one fell swoop.
>And no, you can't just keep the socket open and pass that to the loader
>-- that implies that you can defer the passing for a while, but the web
>server will time out your connection and close it. Conversely, if the
>intent is *not* to hold the "structure" for a while, then why the heck
>have two pieces?

        The way I see it, the _finder_ is responsible for
downloading the file to the local file system, where the
loader requires it to be. The loader turns these raw bits
into a module.

>Both of our proposals guarantee that stuff in sys.path are not
>pathnames. If I insert a "foo.zip" or a
>"http://host.domain.name/pymodules/", then you certainly dont have
>pathnames.
>
>I believe the biggest issue with my proposal is the fact that the values
>are no longer strings.

        That's easy to fix: have a default 'finder' that is used
if the sys.path entry is a string.
-------------------------------------------------------
John Skaller    email: skaller@maxtal.com.au
		http://www.maxtal.com.au/~skaller
		phone: 61-2-96600850
		snail: 10/1 Toxteth Rd, Glebe NSW 2037, Australia