How to run script from interpreter?

[Steven D'Aprano]

On Fri, 30 May 2014 21:46:55 +1000, Chris Angelico wrote:

> On Fri, May 30, 2014 at 9:27 PM, Steven D'Aprano
> <steve+comp.lang.python at pearwood.info> wrote:
>> This is why I'm so adamant that, while REPLs may be permitted to
>> introduce *new* syntax which is otherwise illegal to the Python parser,
>> (e.g. like IPython's %magic and !shell commands) they *must not* change
>> the meaning of otherwise legal Python syntax. It's simply a bad idea to
>> have legal Python code mean different things depending on what
>> environment you're running it in.
> 
> Hmm. I'm not sure that "raises SyntaxError" is any less a part of the
> language's promise than "evaluates to twice the value of x" is. Would
> you, for instance, permit the REPL to define a new __future__ directive,
> on the basis that it's invalid syntax currently?
> 
>>>> from __future__ import magic_command_history
> SyntaxError: future feature magic_command_history is not defined

"from __future__ import ..." is an instruction to the compiler, hence a 
positive language feature. (That's why in source files any such import 
must occur at the top of the file, before any line of code.) It's a 
little unfortunate that bogus __future__ imports raise SyntaxError 
directly rather than some subclass of it, but given that it occurs at 
compile time, and how rarely one is in a position to try catching the 
exception, its understandable that the core devs didn't bother making it 
a separate subclass.

(On the other hand, "import __future__" is a regular import that can 
occur any time you like.)

No, a REPL cannot legitimately invent new language features that change 
Python's semantics, any more than they can legitimately redefine the plus 
operator + to perform subtraction. But they can legitimately add features 
to the shell, provided those features don't affect Python code. What's 
the difference?

Consider entering this part of an interactive IPython session:


In [14]: len []
-------> len([])
Out[14]: 0

In [15]: n = len []
------------------------------------------------------------
   File "<ipython console>", line 1
     n = len []
              ^
SyntaxError: invalid syntax


Why is [14] permitted, but not [15]? Because [14] is a shell feature, but 
[15] is Python code. If [15] were permitted, then we would be tempted to 
use it inside functions:

def spam(a):
    n = len a
    ...

and either be surprised at the syntax error, or (worse!) be surprised 
that the function runs interactively inside IPython but fails in other 
shells and non-interactively. If [15] were allowed, we would no longer be 
running Python code.

Of course, a naive user will be confused that len [] is permitted in 
IPython, but that's okay since it's a power-user's shell aimed at power-
users. If the vanilla Python shell introduced such power-user convenience 
features by default, I would be very disappointed.

Before you ask, there is no absolutely hard and fast line between "shell 
feature" and "Python code", but the more closely your shell features 
resemble Python code, the harder it will be for users (power or not) to 
keep them separate in their head and the more likely they will be 
confused. E.g. suppose my shell decided to allow lines like

var = len []

to define the variable var and set it to 0. I'd argue that crosses the 
line from "useful convenience feature for power-users" to "dangerously 
confusing misfeature" because it looks too much like Python code. (I'm 
already dubious about len [] on its own.) A better interface is to have a 
clear separation between shell commands and Python, and IPython commonly 
usually uses prefix sigils such as ; % and ! for that.

More here, including some cases where I think IPython crosses that line:

http://ipython.org/ipython-doc/dev/interactive/reference.html


You raise the issue of the vanilla Python shell printing the result of 
expressions. That would be the P in REPL, yes? :-) It would be a funny 
REPL that *didn't* print evaluated expressions.

(Not entirely unheard of though -- Forth, for example, doesn't print the 
values you push onto the stack unless you pop them from the stack first. 
But it does print "ok" after each Read-Eval cycle when working 
interactively, so I guess it still counts as a REPL.)

If we wanted to be pedantic, then yes there are semantic differences 
between code running in a REPL and the same running non-interactively. 
The vanilla REPL sets the magic variable _ to the result of the last 
evaluated expression (IPython has a bazillion variations of that). The 
REPL defines sys.ps1 and sys.ps2, when running non-interactively they 
don't exist. PYTHONSTARTUP isn't loaded outside of the REPL. But my 
argument is that these are a different kind of semantic difference than 
changing how Python expressions are parsed. Not all semantic differences 
are equal:

(1) in the REPL, evaluating "(spam . attr)" on its own has the 
    side-effect of printing the value of spam.attr

(2) in the REPL, evaluating "(spam . attr)" does not perform a
    lookup of attr on spam

are very different kinds of behavioural changes.

[For brevity I left out the newlines.]


> I don't think SyntaxError equates to "invitation to make changes".

No, not in general. But SyntaxError does give a convenient opportunity to 
add shell features. If a line of text L is valid Python code, then the 
REPL needs to treat it as valid Python code. If L is invalid Python code, 
i.e. raises SyntaxError, then the REPL *may* use it as a shell feature, 
but whether it should or not depends on the details of L.


-- 
Steven