[Python-Dev] zipimport, round 3 (or would that be that 37?)

[Paul Moore]

Just van Rossum <just@letterror.com> writes:

> Paul Moore wrote:

>> Zipimport doesn't work at all for me as a dynamic extension. Is
>> this just a temporary problem, an issue specific to the zipimport
>> module, or a limitation in the hook mechanism? (I assume not the
>> last, as that would make it pretty useless as a general
>> mechanism...)
>
> I don't know, as you don't give specifics about what error you get,
> if any.  Maybe -v tells us more. (Although zipimport.c can sure use
> better -v reporting.)

Sorry. Basically it just didn't find the module in the zipfile. I
don't think it's a big deal. If you want, I can rebuild as a dynamic
module and try with -v.

>> Do we need to set an error string there? It's not really an error,
>> as such.
>
> I'm not sure. I'd like to make it easy to write hooks, and it makes
> most sense to use classes as import_hooks, and since an __init__
> can't return anything, an exception is the best you can get.

Ah. Now I see the logic (actually, it's the same as iu.py). I'd say
stick with the exception method. What I was really thinking was in the
C code, could I just return -1 without setting the exception. But
that's not good practice, even if it did work.

> It's _possible_ to write a __new__ method that returns None when it
> can't handle the path, but that raises the bar somewhat for writing
> hooks. 

Too right. Leave it as it is.

>> BTW, I don't see any code for checking timestamps of .pyc files
>> against .py files. Did you omit that (and if so, was it deliberate
>> or accidental)?
>
> I copied that verbatim from the PEP 273 patch ;-) I'm sure this
> could be fixed, but there are more important issues at the moment.

It's a fair cop guvnor :-)

To be honest, I think that this is one of the nastiest hurdles to
writing a good import hook - replicating (or even understanding the
logic for!) the hairy extension and timestamp checking code used for
normal imports. Refactoring and encapsulating that would potentially
help in a lot of cases. Except that maybe it's not always exactly the
same (you can't write a compiled .pyc file back to a zip file, you
can't load a C extension from a zipfile, ...) But that isn't an issue
for this patch.

>> I hit an odd case when I was testing interactively. I set sys.path to
>> a list containing just 'ziptest.zip', and then tried to import a
>> module. Zipimport failed because zlib wasn't accessible. That's OK,
>> but when I reset sys.path, zipimport *still* claimed it couldn't
>> decompress as zlib wasn't available. I see why - the optimisation in
>> get_decompress_func() only checks for zlib once - but nevertheless it
>> is annoying that I can only "fix" my mistake by restarting the
>> interpreter. This is never going to happen "in real life", so it's not
>> crucial. But maybe reload(zipimport) could be made to retry the check?
>
> Right, but maybe I simply shouldn't remember that a previous zlib
> import failed, and just try again.

I thought of that - it slows things down when zlib isn't available, as
it checks (and fails) every time. How bad this is, I'm not sure...

>> PS The code *still* looks messy. I think that's just C though - I've
>>    got so used to writing Python code that *any* C looks messy :-)
>
> import.c, zipimport.c or both? ;-)

import.c hasn't got much better - but at least it hasn't got any worse
:-)

> Yeah, it needs work, but you're also right about C: there's a lot of
> string manipulation going on which is a huge pain in the butt. But:
> I admit not very _good_ at doing string manipulations in C, so once
> it's more or less bugfree I'm sure more experienced people can
> improve it if needed.

It wasn't a criticism. Every time I look at C code which manipulates
strings (you're right, that's the bulk of the nastiness) it reminds me
how *nice* virtually any other language is.

But you let me write import hooks in Python and avoid C, so you're my
friend for life :-)

Paul.
-- 
This signature intentionally left blank