[Python-Dev] zipimport, round 3 (or would that be that 37?)

[Just van Rossum]

Paul Moore wrote:

> To get zipimport as a builtin, apply the following patch, and add
> zipimport.c to the pythoncore project.

Ah, thanks, will add that to the next version.

> Zipimport doesn't work at all for me as a dynamic extension. Is this
> just a temporary problem, an issue specific to the zipimport module,
> or a limitation in the hook mechanism? (I assume not the last, as that
> would make it pretty useless as a general mechanism...)

I don't know, as you don't give specifics about what error you get, if any.
Maybe -v tells us more. (Although zipimport.c can sure use better -v reporting.)

> As a builtin, it works fine in a release build. But a debug build
> asserts in fopen() as the filename is "".
> 
> An easy fix for this is to add a check in zipimporter_init(),

Thanks, done.

> Do we need to set an error string there? It's not really an error, as
> such.

I'm not sure. I'd like to make it easy to write hooks, and it makes most sense
to use classes as import_hooks, and since an __init__ can't return anything, an
exception is the best you can get. It's _possible_ to write a __new__ method
that returns None when it can't handle the path, but that raises the bar
somewhat for writing hooks. Lots of stuff in import.c raises ImportError when it
really means: "I can't find what you're asking for". (Btw. I intend to add a
ZipImportError exception to the zipimport module, which would be a subclass of
ImportError.)

> BTW, I don't see any code for checking timestamps of .pyc files
> against .py files. Did you omit that (and if so, was it deliberate or
> accidental)?

I copied that verbatim from the PEP 273 patch ;-) I'm sure this could be fixed,
but there are more important issues at the moment.

> I hit an odd case when I was testing interactively. I set sys.path to
> a list containing just 'ziptest.zip', and then tried to import a
> module. Zipimport failed because zlib wasn't accessible. That's OK,
> but when I reset sys.path, zipimport *still* claimed it couldn't
> decompress as zlib wasn't available. I see why - the optimisation in
> get_decompress_func() only checks for zlib once - but nevertheless it
> is annoying that I can only "fix" my mistake by restarting the
> interpreter. This is never going to happen "in real life", so it's not
> crucial. But maybe reload(zipimport) could be made to retry the check?

Right, but maybe I simply shouldn't remember that a previous zlib import failed,
and just try again.

> I also tried a Python import hook. It seemed to be doing the right
> things. I didn't try too hard - just dummy {find,load}_module methods,
> and an __init__. It looked like errors were generated as a result of
> me not implementing the protocol right, which is good :-)
> 
> Paul.
> 
> PS The code *still* looks messy. I think that's just C though - I've
>    got so used to writing Python code that *any* C looks messy :-)

import.c, zipimport.c or both? ;-) Yeah, it needs work, but you're also right
about C: there's a lot of string manipulation going on which is a huge pain in
the butt. But: I admit not very _good_ at doing string manipulations in C, so
once it's more or less bugfree I'm sure more experienced people can improve it
if needed.

Just