[Python-Dev] confusing exec error message in 3.0

[Guido van Rossum]

On Wed, Aug 27, 2008 at 6:21 PM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Thu, 28 Aug 2008 08:39:01 am Georg Brandl wrote:
>> Fredrik Lundh schrieb:
>> > (using 3.0a4)
>> >
>> >  >>> exec(open("file.py"))
>> >
>> > Traceback (most recent call last):
>> >    File "<stdin>", line 1, in <module>
>> > TypeError: exec() arg 1 must be a string, file, or code object, not
>> > TextIOWrapper
>> >
>> > so what's "file" referring to here?
>> >
>> > (the above works under 2.5, of course)
>>
>> See http://bugs.python.org/issue1762972 -- it has been decided to
>> drop that possibility.
>
> Hmmm... I have a concern with one of the patches in that issue; I refer
> to patch that changes the semantics of module's __file__ attribute.
>
> Guido noted that exec(open(M.__file__).read(), M.__dict__) almost worked
> as a replacement for reload(), except that there were issues with file
> extensions (.py, .pyc, .pyo and even things like .pyc24). So it was
> decided that M.__file__ should always point to the source file.
>
> But now that reload() is now moved into the imp module, I don't see that
> the justification for changing the semantics of M.__file__ still
> exists. imp.reload(M) is much better for interactive use than
> exec(open(M.__file__).read(), M.__dict__).
>
> Is there still a justification for having M.__file__ point to the source
> even if the module was actually loaded from the .pyc version? If so,
> what is that?
>
> Some years ago, as a newbie, I was having trouble with reload()
> repeatedly not picking up changes I was making to a module. The problem
> turned out to be user-error, but how I discovered that was by looking
> at M.__file__ and noticing that Python was loading the .pyc file
> instead of the .py file I was expecting. Had M.__file__ given
> misleading information, I would have been mislead for much longer.
> Here's a small contrived example under Python 2.5:
>
>>>> import parrot
>>>> parrot.__file__
> 'parrot.py'
>>>> # pretend that I made changes to the source
> ... # in my editor, but forgot to save them
> ... reload(parrot)
> <module 'parrot' from 'parrot.pyc'>
>>>> parrot.__file__
> 'parrot.pyc'
>
>
> I don't think M.__file__ should lie and say it was loaded from a file
> that it wasn't loaded from. It's useful to be able to look at a module
> and see what file it was actually loaded from.

While appreciate the use case, there are way more use cases where
there's code that must painstakingly strip the trailing 'c' or 'o'
from __file__ in order to read the source code. Perhaps we should have
a separate API for finding out whether a module was loaded from source
or from a .pyc file; but I think it would be better to have such an
API somewhere in the imp module. It's also possible to follow what
goes on by watching the verbose -v output.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)