[Python-Dev] Proto-PEP regarding writing bytecode files

[Skip Montanaro]

    Paul> This is of particular interest to me, as it will potentially
    Paul> affect how people write import hooks. For example, I can't
    Paul> immediately see how this proposal will interact with the new
    Paul> zipimport module. 

I thought the concensus was that zip files shouldn't normally contain
bytecode anyway.  If so, then perhaps it could always be treated as a
read-only directory.  (Nothing about zip files is in the pep at present.
Feel free to suggest text.  I'm largely unfamiliar with them and haven't
done much to keep abreast of the saga of the import hook.)

In the absence of this proposal, what happens today if a source file is
located in a zip file and there is no bytecode with it?  Does a bytecode
file get written?  If so, is it actually inserted into the zip file?

    Paul> In particular, the PEP needs to say something about what, if any,
    Paul> requirements it places on the writers of import hooks ("If an
    Paul> import hook attempts to cache compiled bytecode, in a similar way
    Paul> to the builtin filesystem support, then it needs to check the
    Paul> environment variable, and...")

I'll save your message, but I'd like to defer this issue for the time being.

    >> - Interpretation of a module's __file__ attribute....

    Paul> Again, this is going to be something import hook writers need to
    Paul> be aware of, and to cater for. Offhand, I can't recall what the
    Paul> import hook PEP considers the fate of imp.find_module to be
    Paul> [... click, click, browse, read...]

    Paul> OK, if you have a module loaded from an import hook, you don't
    Paul> have imp.find_module. You have a new function, imp.get_loader(),
    Paul> but that returns only a loader, not a file name.

I don't understand.  Are you saying imp.find_module()'s results are
undefined or that it simply doesn't exist?

I still believe the module's __file__ attribute should reflect where the
bytecode was found.

    Paul> So you need to expand on this: "If people want to locate a
    Paul> module's source code, they will not be able to except by using
    Paul> imp.find_module(module), which does not take account of import
    Paul> hooks.  Whether this is good enough will depend upon the
    Paul> application".

Feel free. ;-) I'll get to it eventually, but if you can come up with
something plausible I'll gladly use it.

    Paul> [But in any case, PEP 302 points out about __file__ that "This
    Paul> must be a string, but it may be a dummy value, for example
    Paul> "<frozen>"" so anyone using __file__ already needs to be aware of
    Paul> border cases involving non-filesystem modules]

I think this is wrong.  If you can provide a valid way for the user to
locate the bytecode you are obligated to do so.  The only situation where I
think a dummy value is appropriate is if the source code came from a string.

    Paul> Personally, I have no real need for, nor a great interest in, this
    Paul> feature. But others may care a lot about it, and have little need
    Paul> for import hooks. I therefore can't put a good judgement on
    Paul> whether these issues are significant - but I do think the PEP
    Paul> needs to address them, if only to say that the two features don't
    Paul> work well together, and that is understood and not considered to
    Paul> be a significant issue.

I'm not sure I see a lot of conflict, but I'm more likely to be one of those
people who ignores import hooks.  I haven't used one yet, so it's unlikely
I'll start tomorrow. ;-)

Skip