<p dir="ltr"><br>
On 28 Oct 2013 08:41, "PJ Eby" <<a href="mailto:pje@telecommunity.com">pje@telecommunity.com</a>> wrote:<br>
><br>
> On Sun, Oct 27, 2013 at 4:59 PM, Nick Coghlan <<a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a>> wrote:<br>
> ><br>
> > On 28 Oct 2013 02:37, "PJ Eby" <<a href="mailto:pje@telecommunity.com">pje@telecommunity.com</a>> wrote:<br>
> >><br>
> >> On Sun, Oct 27, 2013 at 1:03 AM, Nick Coghlan <<a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a>> wrote:<br>
> >> > Now, regarding the signature of exec_module(): I'm back to believing<br>
> >> > that loaders should receive a clear indication that a reload is taking<br>
> >> > place. Legacy loaders have to figure that out for themselves (by<br>
> >> > seeing that the module already exists in sys.modules), but we can do<br>
> >> > better for the new API by making the exec_module signature look like:<br>
> >> ><br>
> >> > def exec_module(self, module, previous_spec=None):<br>
> >> > # module is as per the current PEP 451 text<br>
> >> > # previous_spec would be set *only* in the reload() case<br>
> >> > # loaders that don't care still need to accept it, but can<br>
> >> > just ignore it<br>
> >><br>
> >> Just to be clear, this means that a lazy import implementation that<br>
> >> creates a module object without a __spec__ in the first place will<br>
> >> look like an initial import? Or will that crash importlib because of<br>
> >> a missing __spec__ attribute?<br>
> >><br>
> >> That is, is reload()'s contract adding a new prerequisite for the<br>
> >> object passed to it?<br>
> >><br>
> >> (The specific use case is creating a ModuleType subclass instance for<br>
> >> lazy importing upon attribute access. Pre-importlib, all that was<br>
> >> needed was a working __name__ attribute on the module.)<br>
> ><br>
> > For custom loaders, that's part of the contract for create_module() (since<br>
> > you'll get an ordinary module otherwise),<br>
><br>
> Huh? I don't understand where custom loaders come into it. For that<br>
> matter, I don't understand what "get an ordinary module object" means<br>
> here, either.<br>
><br>
> I'm talking about userspace code that implements lazy importing<br>
> features, like the lazyModule() function in this module:<br>
><br>
> <a href="http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup">http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup</a><br>
><br>
> Specifically, I'm trying to get an idea of how much that code will<br>
> need to change under the PEP (and apparently under importlib in<br>
> general).</p>
<p dir="ltr">If the lazy import is injected by a *different* module, then nothing changes.</p>
<p dir="ltr">> > and so long as *setting* the<br>
> > special module attributes doesn't cause the module to be imported during the<br>
> > initial load operation, attribute access based lazy loading will work fine<br>
> > (and you don't even have to set __name__, since the import machinery will<br>
> > take care of that).<br>
><br>
> There's no "initial load operation", just creation of a dummy module<br>
> and stuffing it into sys.modules. The way it works is that in, say,<br>
> foo/__init__.py, one uses:<br>
><br>
> bar = lazyModule('foo.bar')<br>
> baz = lazyModule('foo.baz')<br>
><br>
> Then anybody importing 'foo.bar' or 'foo.baz' (or using "from foo<br>
> import bar", etc.) ends up with the lazy module. That is, it's for<br>
> lazily exposing APIs, not something used as an import hook.</p>
<p dir="ltr">I was thinking of the more complex case where a module causes *itself* to be loaded lazily. Nothing changes for the simpler case where the injection occurs in a different module.</p>
<p dir="ltr">> > For module level lazy loading that injects a partially initialised module<br>
> > object into sys.modules rather than using a custom loader or setting a<br>
> > __spec__ attribute, yes, the exec_module invocation on reloading would<br>
> > always look like a fresh load operation (aside from the fact that the custom<br>
> > instance would already be in sys.modules from the first load operation).<br>
><br>
> Right.<br>
><br>
><br>
> > It *will* still work, though (at least, it won't break any worse than such code<br>
> > does today, since injecting a replacement into sys.modules really isn't<br>
> > reload friendly in the first place).<br>
><br>
> Wait, what? Who's injecting a replacement into sys.modules? A<br>
> replacement of what? Or do you mean that loaders aren't supposed to<br>
> create new modules, but use the one in sys.modules?<br>
><br>
> Honestly, I'm finding all this stuff *really* confusing, which is kind<br>
> of worrying. I mean, I gather I'm one of the handful of people who<br>
> really understood how importing *used to work*, and I'm having a lot<br>
> of trouble wrapping my brain around the new world.</p>
<p dir="ltr">Provide test cases that exercise the situations you're concerned about supporting and then you don't need to worry about them breaking.</p>
<p dir="ltr">><br>
> (Granted, I think that may be because I understand how a lot of old<br>
> corner cases work, but what's bugging me is that I no longer<br>
> understand how those old corners work under the new regime, nor do I<br>
> feel I understand what the *new* corners will be. This may also just<br>
> be communication problems, and the fact that it's been months since I<br>
> really walked through importlib line by line, and have never really<br>
> walked through it (or PEP 451) quite as thoroughly as I have import.c.<br>
> I also seem to be having trouble grokking why the motivating use<br>
> cases for PEP 451 can't be solved by just providing people with good<br>
> base classes to use for writing loaders -- i.e., I don't get why the<br>
> core protocol has to change to address the use case of writing loaders<br>
> more easily. The new protocol seems way more complex than PEP 302,<br>
> and ISTM the complexity could just be pushed off to the loader side of<br>
> the protocol without creating more interdependency between importlib<br>
> and the loaders.)</p>
<p dir="ltr">Bad loader implementations have too much power to break the import system, and loaders don't currently expose enough information about modules that *could* be imported.</p>
<p dir="ltr">We also take "inherit from this class to implement the protocol correctly, import will break if you don't" as a sign that the *protocol is broken* by pushing too much of the work onto the plugins.</p>
<p dir="ltr">New style loaders can be much simpler, because the import system takes care of the complexity, without relying on users inheriting from a particular base class.</p>
<p dir="ltr">The complexity comes from the fact that we're now breaking down what the *real* expectations on a loader actually are, and making those part of the import system itself.</p>
<p dir="ltr">Cheers,<br>
Nick.<br>
</p>