[Python-Dev] Reading Python source file

[Guido van Rossum]

On Thu, Nov 19, 2015 at 1:47 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 17.11.2015 16:22, Guido van Rossum wrote:
>> On Tue, Nov 17, 2015 at 1:59 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>>> [moving from read source line by line to reading all in one go]
>>> We use the same simplification in eGenix PyRun's emulation of
>>> the Python command line interface and it has so far not
>>> caused any problems.
>>
>> Curious how you do it? I'd actually be quite disappointed if the
>> amount of parsing done by the standard REPL went down.
>
> Oh, that's easy:
>
>         elif sys.argv[0] == '-' and not (pyrun_as_string or pyrun_as_module):
>             # Read the script from stdin
>             pyrun_as_string = True
>             pyrun_script = sys.stdin.read()
>
> and then, later on:
>
>         # Run the script
>         try:
>             pyrun_execute_script(pyrun_script, mode)
>         except Exception as reason:
>             if pyrun_interactive:
>                 import traceback
>                 traceback.print_exc()
>                 pyrun_prompt(banner='')
>             else:
>                 raise
>         else:
>             # Enter interactive mode, in case wanted
>             if pyrun_interactive:
>                 pyrun_prompt()

Yes, this makes sense.

> The REPL is not affected by this, since we use the standard
> code.interact() for the prompt. This reads the entry line
> by line, joins the lines and tries to compile the entry every
> time it receives a new line until it succeeds or fails.

Heh, I wrote code.interact() as a poor-man's substitute for what the
"real" REPL (implemented in C) does. :-) It usually ends up doing the
same thing, but I'm sure there are edge cases where the "real" REPL is
better. It doesn't re-parse after each line is read, it actually keeps
the parser state and adds new tokens read from the tty. There is even
a special grammar root symbol ('single') for this mode.

> Serhiy's proposed change should not affect this mode of
> operation.

I sure hope not.

Though there is actually one case that IIRC doesn't work today: if
sys.stdin is a stream that doesn't wrap a file descriptor. Would be
nice to make that work. (Pretty esoteric use case though.)

-- 
--Guido van Rossum (python.org/~guido)