[Python-ideas] Syntax: 'return: ...' expressions

Andrew Barnert abarnert at yahoo.com
Sat Jan 24 12:06:52 CET 2015

On Jan 23, 2015, at 23:53, Ron Adam <ron3200 at gmail.com> wrote:

I fear we're probably getting pretty far off-topic for python-ideas, but I'm going to respond anyway.

> On 01/24/2015 12:06 AM, Andrew Barnert wrote:
>> On Friday, January 23, 2015 2:06 PM, Ron Adam<ron3200 at gmail.com>
>> wrote:
>>>> On 01/23/2015 08:56 AM, Nick Coghlan wrote:
>>>>>> I really like that "each statement is like a single step in a
>>>>>> mathematical proof" analogy.
>>>> I agree.
>>>> There are some other distinctions.  When you consider these, you can
>>>> see how many language designers come to similar solutions.
>>>> Expressions evaluate in unique name spaces, while statements
>>>> generally do not. Consider "a + b"; it is evaluated in a private
>>>> method after the values a and b are passed to it.
>> I don't think that's true.
> Not all the time, which is why I said "generally". :-)
>> Consider "a[b] = c", which is a statement,
>> but it's evaluated in a private method after the values a, b, and c are
>> passed to it. The fact that it's a.__setitem__ rather than a.__add__
>> doesn't seem particularly important. I think the key is that __setitem__
>> doesn't have a return value--like (nearly) everything in Python that
>> mutates state--and therefore there's no expression for it. The question
>> is, what does that buy?
> Right, and good question, but consider that it's not adding or altering the name space the 'a' object is in.  It's a convenience/exception for using objects.

I think that's the point. In theory, objects are just syntactic sugar for closures; in practice, objects are an intuitively useful way to represent mutable state. And the fact that things that mutate an object are statements (whether directly, because they're __setattr__ type calls, or indirectly, because they're expressions that return None and can therefore only be used in expression statements) is important in making Python readable in practice.

> If objects were done with closures, then the same modification would be done with an assignment statement.  And then the generalisation would be more consistent.  But at a cost is other ways.
>>>> Statements are used to mutate the current name space, while
>>>> expressions generally do not.
>> This is closer. I think, more generally, statements are used to mutate
>> the important state that the local function is all about mutating. And
>> the thing that gets mutated is almost always the leftmost thing. That
>> definitely helps in scannability.
> I'm not quite sure I follow that.  I think what you are calling the important state, I think of as the shared state. A value that will be used multiple times in the same frame.

By "important state", I just mean whatever state the reader is likely to care about. And such state is almost always modified by a statement with some readable identification on the left side--whether it's a global/closure/local assignment, an attribute assignment, an element assignment, and augmented version of any of the above, a method call that doesn't return self (and will therefore only be used in an expression statement). So each statement means (at most) one state transition. And it's almost always the leftmost thing that's affected. And I think that aids readability.

> Binding a name to a value mutates the names pace, but it does not mutate the name.  It's still the same name.
> Again this can be view multiple way if you consider how it actually works.  Some languages use a stack to implement a name. And in a new frame, a new bound value would get pushed on the name stack.  I'm not completely sure that python doesn't do something like that in some places to speed thing up.  But I don't think so.

Sure, pure non-mutating languages can use a stack to implement bindings. But in a mutating language, that doesn't work if you have closures. CTM explains what you get out of mutable state (and what it costs) very nicely. Of course in practice any Python implementation has to be able to detect whether a given scope might have closures referring to it, so it _could_ switch to rebinding using a bindinmg stack, but in practice any Python implementation is likely to convert local accesses to offsets as CPython does, which is a much better optimization (at least for a language where rebinding is idiomatically common) that precludes that option.

>>>> Statements can alter control flow, while expressions generally do
>>>> not.
>> Sure, but I think this part only really helps if control flow is somehow
>> visible. In Python, it is, because control flow almost always means
>> compound statements, which means indentation, and very little else means
>> indentation.
> You are referring to the visual aspect, while I'm referring to what a statement does.  Same thing. ;-)

Well, I'm highlighting the visual aspect because I think that's important to Python's readability, and to why statements contribute to that readability. If statements and expressions had similar indentation rules (as in CoffeeScript), I don't think Python would get the same benefit from having statements.

>>>> Having a clear distinction between expressions and statements makes
>>>> reading and understanding code much easier.
>> Definitely. That's the part I think is key, but am struggling to
>> explain.
>> I think Guido offers a great analogy in mathematical proofs, but the
>> question is to find the actual commonality behind the analogy.
> Form follows function...  Just one way to look at it.  Sometimes what something does can come from the shape it has too.

Yes. But sometimes the shape is limiting, rather than expanding--and yet that limitation itself can be used to add meaning. (See Guido's point about code that fits in a window/screen/page.)
So function partly follows form. And mathematical proofs are a great example. The fact that there are a limited number of ways you're allowed to get from the previous statements to the next one makes each step more readable.

>> After some more thought on this, I think what it comes down to is that
>> (idiomatically-written) Python lets you skim the control flow (because
>> all non-trivial control flow, and very little else, is expressed in
>> terms of indentation) and the state transitions (because each statement
>> generally mutates at most one thing, and it's the leftmost thing), so
>> you can quickly find the part of the code you actually need to read
>> carefully, instead of having to read the whole thing.
>> I've written this idea up in a bit more detail here:
>> http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.html
> Very interesting.. and thanks for the mention.  ;-)

Sure; as I said, your last paragraph (well, the last one I quoted) puts the whole thing I'm trying to answer much more clearly than I've been able to.

> One of the things that makes a difference is to be able to hold a simplified model in your mind while you are working on it.  

Yes! That's something else that was on the tip of my tongue that I couldn't explain clearly. Being able to hold enough of the syntax in your head to parse code subconsciously (which CoffeeScript lacks, as Guido pointed out) is part of it, but you're right, the big deal is being able to hold the entire model in your head.

And at a different level, that's what makes scanability of flow control and state changes so important, which I couldn't explain before. That's the model of an imperative-style function that you need to be able to internalize to understand the function holistically.


> The separation of statements and expressions definitely helps me with that.  If I can easily keep the name space in my mind... or at least the part of it that correspond with the block of code I'm looking at, It really helps me to visualise things in a meaningful way and see what effect the statements will have on the name space.

Exactly. Which is why the "one obvious mutation per line (usually)" property is so important.

But I think describing it purely in terms of the local namespace hides the fact that it applies just as well to OO-style code (where most mutation is to the namespace of self or one of the other parameters) as to traditional structured imperative code.

> When statements and expressions don't represent what they do in an obvious way, then all bets are off.  It becomes a mental stumbling block.

> Much of this became clear to me when I wrote a simplified script language that adds statements to a mini scheme like language.  I did need to use braces for blocks.  Even though I didn't need to use visual indentation, I still preferred formatting the programs in a similar style to python.  In this mini language, the separation of statements and expressions is even more pronounced because all expressions are s-expressions, and all statements are not expressions.
> And going one bit further, I used dictionaries for names spaces, like python, but used lists for statement blocks.  So statement blocks have order, but name spaces don't.
>>>> I believe Python follows most of these conventions in most places,
>>>> and when it doesn't, it's usually for a practical reason that are
>>>> fairly obvious.
>>>> For example, an "or" expression is a bit of both.
>>>> Another example of how python chooses a practical alternative is we
>>>> can nest expressions instead of using "call" statements on separate
>>>> lines and a stack to hold the augments and return values.  That is
>>>> what python does in the byte code so we don't have to do it in
>>>> explicit statements.
>>>> If you factor out all expressions you get byte code.  Or if you
>>>> factor out all statements you get something like lisp.
>> I don't think either part of that is true.
> It wasn't meant to be taken absolutely literally.  Which is why I said... "something like".  

OK, but I don't think it's figuratively true in any useful sense either. Languages like Ruby prove that removing statements doesn't have to mean something like Lisp. We aren't stuck with the models of the 60s. So the question of what you give up by factoring out all statements turns out to be more complicated (and more interesting) than it was in those models.

> There is quite a bit of wiggle room when it comes to how different people think about things, and what words we use to describe them.  Some times the hardest part of a discussion is getting the same mental picture in more than one person.  :-)
>> Bytecode is full of things
>> that are expressions—even your paradigm case of an operator expression
>> is handled by an opcode.
> I see opcodes as being similar to keywords that are used in a statement form.  And I view the stack as being a temporary value space.  Of course, I know certain combinations of several opcodes together may correspond to a particular python expression, but individually, each byecode along with it's following few values, are byte code statements to me.
> And even if a single bytecode was the equivalent of a python expression, my point was you can use statements replace statements with expressions.

I don't know what you meant here. Maybe that a stack machine language has a fixed, non-extensible set of expressions, but its set of statements can be effectively arbitrarily extended with jsr/ret to other bytecode? If so, I'll buy that, but I'm not sure how it's relevant. I don't know of any examples of readable expression-free code that compare to such examples of readable statement-free code as, say, anything written in OCaml, or any Ruby code that sticks to single-return style. Eliminating expressions is clearly a non-starter for a readable language; eliminating statements is actually arguable--and the whole point is to find the arguments against doing so.

> Byte code is a good example of that.  But you still have functions calls.. You just can't nest them in the same way you do with python function calls, you must push them on the stack and use CALL "statements" to execute it, and then use another statement to store the value that gets put on the top of the stack.  (or push another function on the stack that will use that value...)
>> And conversely, CoffeeScript (if you avoid
>> break/continue/return statements) factors out all statements, and Ruby
>> comes close to doing so, and yet they're really not more Lisp-like than
>> Python in any meaningful way.
> See the next paragraph... ;-)
>>>> Of course this subject is definitely a very subjective one which
>>>> relays on agreeing on the general meaning of the above sentences.
>>>> OR... YMMV.

Sure, but I'm not sure I understand the meaning you're going for.

And, more importantly, I think there is an objective sense in which Python uses statements to gain subjective readability, and that objective sense is something we can question and try to answer.

More information about the Python-ideas mailing list