[Python-ideas] __loader__.get_source(): semantics of returning None

[Paul Moore]

On 28 April 2018 at 17:09, Jeroen Demeyer <J.Demeyer at ugent.be> wrote:
> The documentation of __loader__.get_source() says
>
> "Returns None if no source is available (e.g. a built-in module)."
>
> But what does "no source is available" mean precisely? It could mean either
> of two things:
>
> (A) I am absolutely certain that there is no source anywhere to be found.
>
> (B) I don't know where to find the source, but if you look hard enough, you
> may find it.

The intention was (B). But to an extent, that misses the point. If you
only have bytecode on your system, then the loader can't find the
source - but *of course* there is source "somewhere", probably not on
your system, though. "Looking hard enough" isn't really well defined.
If you consider this in terms of "what the loader knows" then there's
no difference between (A) and (B).

> Currently, linecache interprets it as (A). Is there any chance that we can
> either change the interpretation for returning None to (B) or to provide an
> officially documented way to answer (B)? This could be using a new return
> value (say, NotImplemented) or by not implementing get_source at all (such
> that __loader__.get_source raises AttributeError). The latter is probably
> how things already work in practice, but it isn't really documented that
> way.
>
> The context for this question/proposal is https://bugs.python.org/issue32797

I've never used linecache, but that sounds a bit weird. I've just
looked at the docs:

"""
If a file named filename is not found, the function will look for it
in the module search path, sys.path, after first checking for a PEP
302 __loader__ in module_globals, in case the module was imported from
a zipfile or other non-filesystem import source.
"""

That sounds like exactly what is being requested in the issue - so I'd
say this is simply a bug in linecache, for not continuing with the
path search after the loader check fails. It's not a problem with
"interpreting" the loader's result, but simply a case of not doing
what the docs say it does...

> When the linecache module is asked for the source code of a certain file, it
> queries the __loader__.get_source() for the source code. If this returns
> None, that's the end: no source is returned.

But that's not what the linecache docs (quoted above) say that it does
- so this is a straight bug in linecache.

> However, linecache is also passed the filename! So even if it could find the
> source by filename, it won't even try that if get_source() returns None.

And that's also a bug - the docs explicitly say that the filename is
tried first.

Paul