[Patches] [ python-Patches-652586 ] New import hooks + Import from Zip files

[noreply@sourceforge.net]

Patches item #652586, was opened at 2002-12-12 11:34
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=652586&group_id=5470

Category: Core (C code)
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Just van Rossum (jvr)
Assigned to: Nobody/Anonymous (nobody)
Summary: New import hooks + Import from Zip files

Initial Comment:
This patch implements two things:
- a new set of import hooks, modelled after iu.py
- builtin support for imports from Zip archives (a
competing implementation for PEP 273)

The new set of hooks probably need a better document
explaining them (perhaps a PEP). My motivations have
been posted to python-dev.

Here's a brief description.

Three new objects are added to the sys module:
- path_hooks
- path_importer_cache
- meta_path

sys.path_hooks is a list of callable objects that take
a string as their only argument. A hook will be called
with a sys.path or pkg.__path__ item. It should return
an "importer" object (see below), or raise ImportError
or return None if it can't deal with the path item. By
default, sys.path_hooks only contains the zipimporter
type, if the zipimport module is available.

sys.path_importer_cache is a dict that caches the
results of sys.path_hooks to avoid repeated hook lookups.

sys.meta_path is a list of importer objects that are
invoked *before* the builtin import mechanism kicks in.
This allows overriding of builtin module and frozen
module import, but the main feature is that it allows
importer objects *without* a corresponding sys.path
item (just like builtin and frozen modules).

Importer objects must conform to the following protocol:

i.find_module(fullname) -> None or an importer object
i.load_module(fullname) -> the imported module (or
raise ImportError)

The 'fullname' is always the fully qualified module
name, ie. a dotted name for a submodule.

This patch adds one more feature: a sys.path item may
*itself* be an importer object. This is convenient for
experimentation, but using it may break third-party
code that assumes sys.path contains only strings.

----------------------------------------------------------------------

>Comment By: Just van Rossum (jvr)
Date: 2002-12-12 13:39

Message:
Logged In: YES 
user_id=92689

Hm, regarding gc again: zipimporter objects can only
*theoretically* be involved in cycles, and only if people
muck with the "files" attribute. As it is, the "files" dict
only contains strings (keys) and tuples (values) which
contain strings and ints. So is it really neccesary?

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-12-12 13:35

Message:
Logged In: YES 
user_id=92689

Martin:
- Have you actually seen a recursive import of zlib? I don't
see how it's possible to cause a stackoverflow. Hm, maybe if
someone stuffs a zlib.py in a zip archive? I'll add a guard.
- Are you really saying it *should* be a builtin module, or
was that comment supposed to start with "if"? See also
Paul's remark about zlib: if zlib remains a shared lib,
having zipimport as a builtin only helps for non-compressed
archives.
- gc: I'll give this a go (never done it before!)
- get_long() -> it's called with a signed char *, and the
buf[i] & 0xff should take care of the rest, so I'm not sure
I understand. But trust your judgement and changed it in my
working copy.

Thank you very much for the quick reply!

----------------------------------------------------------------------

Comment By: Paul Moore (pmoore)
Date: 2002-12-12 12:23

Message:
Logged In: YES 
user_id=113328

I can provide details on how to patch the Windows build 
process - it's not hard. The question of whether to have 
zipimport as builtin or dynamic should be resolved first (the 
processes are different in the 2 cases).

It should be pointed out that if zipimport is builtin, it still relies 
on a dynamic module (zlib) for importing from compressed 
zips. I don't know whether this affects the decision, though...

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2002-12-12 12:15

Message:
Logged In: YES 
user_id=21627

Some comments:
- zipimport should normally be built as a builtin module, there
  should also be a patch for the Windows build procedure
- zipimporters need to support garbage collection, as they 
can occur in cycles
- there should be a mechanism to prevent a stack overflow in 
case of recursive import of zlib.
- get_long needs to accept an unsigned char*

----------------------------------------------------------------------

Comment By: Just van Rossum (jvr)
Date: 2002-12-12 11:37

Message:
Logged In: YES 
user_id=92689

btw. the attached file contains a patch for various files as
well as a new file: zipimporter.c. Place the latter in the
Modules/ directory.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=652586&group_id=5470