On Fri, Oct 25, 2013 at 2:10 PM, PJ Eby <pje@telecommunity.com> wrote:
On Fri, Oct 25, 2013 at 1:15 PM, Brett Cannon <brett@python.org> wrote:
>
> On Fri, Oct 25, 2013 at 12:24 PM, PJ Eby <pje@telecommunity.com> wrote:
>> At least through all 2.x, reload() just uses module.__name__ to
>> restart the module find-and-load process, and does not assume that
>> __loader__ is valid in advance.
>
>
> That doesn't make much sense in a post-importlib world where import makes
> sure that __loader__ is set (which it has since Python 3.3). Otherwise you
> are asking for not just a reload but a re-find as well.

That's a feature, not a bug.  A reload() after changing sys.path
*should* take into account the change, not to mention any changes to
meta_path, path hooks, etc.  (And it's how reload() worked before
importlib.)

Fair enough, but in my mind that obviously doesn't really click for what I view as a reload in an importlib world where secret import code no longer exists. When I think re-load I think "load again", not "find the module again and execute a load with a possibly new loader".

It's not worth changing in Python 3.3, but if you want to propose to redefine importlib.reload() to use importlib.find_spec() to reset the __spec__ on a module then I'm not going to argue against it (but I'm not going to fight for it either).

And in a PEP 451 world it should be dead-simple to make this work the way you want in your own code even if this doesn't go the way you want::

  spec = importlib.find_spec(name)
  module.__spec__ = spec
  importlib.reload(module)  # Which in itself is essentially init_module_attrs(spec, module); spec.loader.exec_module(module)

Heck, you can do this in Python 3.3 right now::

  loader = importlib.find_loader(name)
  module = sys.modules[name]
  module.__loader__ = loader
  importlib.reload(module)


I suppose it's not really documented all that well,

Nope =)
 
but way way back
in the 2.3 timeframe I asked for a tweak to PEP 302 to make sure that
reload() (in the re-find sense) would work properly with PEP 302
loaders like zipimport -- some of the language still in the PEP is
there specifically to support this use case.  (Specifically, the bit
that says loaders *must* use the existing module object in sys.modules
if there's one there already, so that reload() will work.  It was
actually in part to ensure that reload() would work in the case of a
re-find.)

Ah, okay. That is not explicit in the PEP beyond coming off a total nuisance in order to support reloading by the loader, not an explicit finder + loader use-case.
 

It appears that since then, the PEP has been changed in a way that
invalidates part of the purpose of the prior change; I guess I missed
the discussion of that change last year.  :-(

ISTM there should've been such a discussion, since IIRC importlib
wasn't supposed to change any Python semantics, and this is a
non-trivial change to the semantics of reload() even in cases that
aren't doing lazy imports or other such munging.

Well, not change them to the best of its abilities. I'm sure there are edge cases that I couldn't match properly since I could only go by the test suite and bug reports filed since Python 3.1.
 
 reload() used to
take sys.* and __path__ changes into account, and IMO should continue
to do so.

Unfortunately this was never explicitly documented (at least that I noticed) nor was there a test in the stdlib to enforce compliance with it, else this never would have happened.
 
 If this is an intentional change in reload() semantics,
other Python implementations need to know about this too!

Not really. As of Python 3.3 the semantic shift included re-implementing the function in pure Python so they pick up the change for free; remember the function is not a builtin anymore to somewhat de-emphasize its use since it's kind of a hack and really only used typically at the interpreter prompt as an easy part of a load-edit-reload dev cycle, not some fundamental operation.
 

(That being said, I'm not saying I shouldn't or couldn't have tested
this in 3.3 and found out about it that way.  And the existence of
issue18698 suggests that nobody's relying yet on even the
*fundamental* semantics of PEP 302 reload() working properly in 3.3,
since that was an even bigger change that nobody spotted till a couple
of months ago.)

Yep, never got a bug report on this (although there have been reports of typos in the docs as well as not returning what's in sys.modules so it is being used by people).