On Fri, Nov 1, 2013 at 7:35 AM, Brett Cannon <brett@python.org> wrote:
> On Thu, Oct 31, 2013 at 6:10 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:

>> At the time when find_spec runs, sys.modules hasn't been touched yet, so
>> the old module state is still available when reloading. Passing the spec in
>> lets the loader decide whether or not it can actually load that module given
>> the information about the previous load operation.
>
> Do you mean loader or finder? You say "find_spec" which is for finders, but
> then you say "loader".
>
>>
>> However, perhaps it makes more sense to pass the entire existing module,
>> rather than just the spec? I'd like to minimise the need for new-style
>> loader authors to retrieve things directly from the sys module, and you're
>> right that "can I reload into this particular module?" is a slightly
>> different question from "can I reload a module previously loaded using this
>> particular spec?".
>>
>> A spec-based API could still be used to answer the first question by
>> looking up sys.modules, but so could the name based API. Passing in the
>> module reference, on the other hand, should let loaders answer both
>> questions without needing to touch the sys module.
>>
>> I also now think this pass-the-module approach when finding the spec
>> approach could also clean up some of the messiness that is __main__
>> initialisation, where we repeatedly load things into the same module object.
>
>
> See you mention "finding" but before now everything has been "loader".
>
>>
>> Let's say we be completely explicit and call the new parameter something
>> like "load_target". If we do that then I would make the *same* change to
>> runpy.run_path and runpy.run_module: let you pass in an existing module
>> object under that name to say "hey, run in this namespace, rather than a
>> fresh one". (Those APIs currently only support pre-populating a *fresh*
>> namespace, rather than allowing you to directly specify an existing one)
>>
>> Most loaders won't care, but those that do will have all the info they
>> need to throw an exception saying "I could provide a spec for this, but it's
>> not compatible with that load target".
>
> Exactly what APIs -- new or changed -- are you proposing? Method signatures
> only please to avoid ambiguity. =)

I'm about to update the PEP but I'll clarify here.  The signature of
finder.find_spec() will grow an extra "existing" parameter (default to
None) that is an existing module.  During reload the module from
sys.modules will be passed in.  Having this be part of find_spec() is
consistent with the job of the finder to identify a loader that should
be used to load the module (reload or not).

As to the inconsistency in what Nick said, it still fits in with the
way I see it.  I'm sure he'll correct me if he had something else in
mind. :-)  finder.find_spec() will be making a decision on what loader
(if any) to return.  As part of that, when it finds a loader it can
further decide on whether or not the loader supports loading into an
existing module (if one was provided).  As part of that it may or may
not consult with the loader.  Either way it stands as proxy for the
loader in the latter decision.

In lieu of an explicit loader.supports_reload() method, a loader will
rely on its associated finder to be a proxy for reload-related
decisions, likely communicated via spec.loader_state.  Perhaps Nick
meant something else, but my understanding is that he was referring to
this