[Python-Dev] Complexity of import.c

[James C. Ahlstrom]

Martin v. Löwis wrote:
>>I see that my patch to import.c has been criticized
>>as adding to the complexity of an already complex module.

> This is not the case. Much of the complexity of the patch stems from an
> (atleast perceived) attempt to reorganize import.c. For example:

I think package imports really are to blame, because they require
the zip import mechanism to be within import.c.  Remember that
PEP 273 semantics require that a zip archive is equivalent to
a directory hierarchy, not just a single directory.  Then this
must work on __path__:
      /a/b/c/myarchive.zip/subdir/
And consider a package import which changes the path, and then
imports subpackages, each of which may also alter the path.
I don't see how you can jump out of import.c on a sys.path item
and make this work, because the new path is coming from package
import, not the original sys.path.

> - why did is_builtin have to go?

I introduced a dictionary of zip file contents to avoid
repeated linear searches of the zip file.  Since builtins
are a constant list, it was easy to add builtin names too.
I was not necessary to ditch is_builtin, but it was more
natural to do so.

> - Was the ExternalNames dictionary really necessary?

Yikes!  that is where I record the zip file contents.  That
avoids the linear search.

OK, so once I had a dictionary of zip names, it was natural
to add other info, such as directory file names too.

> - Why was the HAVE_READLINK chunk moved?
> - What is the purpose of Get/SetScriptName?

These changes and several others were necessary to be
able to import the Python standard lib from a zip file.
The problem was the attempt to import site.py before the
sys.path was fully set up.  This is arguably a bug in
Python.  The changes moved all sys.path setup to getpath
and out of sysmodule.c.

This isn't an issue of zip importing per se.  It is the
avoidance of phase errors during start up.  For example,
you need a full valid sys.path to be able to import zlib.pyd
(on Windows) so you read a compressed zip archive.  Then
you import site.py plus whatever it imports.  This must be
available in the zip standard lib file, so zip importing
must already work.  Thus the need for careful bootstrapping.

JimA