On Mon, Jan 5, 2015 at 8:49 PM, Andrew Barnert wrote:

Finally, are you sure your new return: isn't ambiguous without look ahead or context? When the parser reads "if spam: return", is it starting a return simple statement, or an expression statement that starts with a return expression? Have you tried actually writing out the grammar?

The way I see it, the return-expression is "return:", whereas the simple return statement would have no colon. Your other concerns, though, are quite valid. ChrisA

On Mon, Jan 5, 2015 at 9:37 PM, Andrew Barnert wrote:

Are you suggesting that "return:" would be a single token (as opposed to all the other colons in the tokenizer), or that you'd use look ahead to disambiguate?

Oh, I see what you mean. I'm sure this can be dealt with, though; if nothing else, requiring that this kind of expression be parenthesized would do it, as 'return' inside parens has no meaning. It'd be like a genexp. But I still don't think it's a good idea :) ChrisA

On Jan 6, 2015, at 2:59, Yawar Amin wrote:

On 2015-01-05 04:49, Andrew Barnert wrote:

...

You also seem to have invented a rule that a sequence of statements has the value of the last statement as its value. Which requires first inventing a rule that gives values to statements. I'll assume you wanted to go with the same rule the interactive interpreter uses to display a last value--an expression statement has its expression's value, and any other kind of statement has None?)

No, I was trying to say that whatever's inside the 'return: ...' block is evaluated, and then the last expression inside the block becomes the value of the block as a whole. No change would be required to any existing expressions or statements, or to the result of a normal sequence of statements.

I think you're missing another important point here: statements and expressions are different things. Blocks are made up of statements, not expressions, and statements don't have values. This is different from many other languages like C, Ruby, or JavaScript, where as many things as possible are expressions (and therefore have values), so a block is (oversimplifying a bit) just a sequence of expressions separated by semicolons and stuck inside braces, and therefore it has a last value. One type of statement, the expression statement, is just an expression on a line by itself (or between semicolons), so you _could_ invent a rule pretty easily that expression statements have the value of their expression, and all other statements have a value of None, and a block has the value of its last statement. This is basically the same rule used by the interactive REPL for displaying the repr of what you've typed and setting the _ variable, so it wouldn't be a huge stretch to add the same rule to the language itself--but it would still be a new rule, and a significant change to the internals of the interpreter, even though it would only be useful in this new context.

LOn Jan 6, 2015, at 14:35, Eugene Toder wrote:

On Tue, Jan 6, 2015 at 3:25 AM, Andrew Barnert wrote:

...

I think you're missing another important point here: statements and expressions are different things. Blocks are made up of statements, not expressions, and statements don't have values. This is different from many other languages like C, Ruby, or JavaScript, where as many things as possible are expressions (and therefore have values), so a block is (oversimplifying a bit) just a sequence of expressions separated by semicolons and stuck inside braces, and therefore it has a last value. Statements and expressions are separate in C and Javascript,

Well, obviously; otherwise "as many things as possible are expressions" would be pointless. In C, unlike Python (and, more to the point, unlike many of C's predecessors), assignment, augmented assignment, and increment are expressions. Different languages have since taken that farther: Ruby makes most flow control into expressions, JavaScript makes function definitions into expressions (which means you can wrap any statement in an expression by defining and calling a function around it), CoffeeScript makes everything but return, continue, and break into expressions.

...

One type of statement, the expression statement, is just an expression on a line by itself (or between semicolons), so you _could_ invent a rule pretty easily that expression statements have the value of their expression, and all other statements have a value of None, and a block has the value of its last statement. This is basically the same rule used by the interactive REPL for displaying the repr of what you've typed and setting the _ variable, so it wouldn't be a huge stretch to add the same rule to the language itself--but it would still be a new rule, and a significant change to the internals of the interpreter, even though it would only be useful in this new context. I don't think one needs to invent this rule in the context of this idea. All that is needed is to end the "return expression" (or whatever you want to call it) with an expression rather than a statement (i.e. return_expr ::= "keyword" statement* expression). The value of this last expression is the value of the whole thing.

That's equivalent to arguing that function bodies don't need return, break, or continue, because you can theoretically write anything to fall off the end. I'm assuming that the OP wanted something that works intuitively similar to other languages, the REPL, etc.--e.g., if the last statement executed is in the true block of an if/else, it doesn't matter that it's not textually the last statement.

On 7 January 2015 at 15:29, Yawar Amin wrote:

Not exactly, I think ... I just wanted a way to 'wrap' statements inside expressions. That would unify the two concepts in Python

The data modelling and control flow aspects of Python, as represented by statements, are deliberately distinct from the computational aspects, as represented by expressions. Some constructs blur the boundaries between computation, data modelling and control flow, and hence may exist in both statement and expression forms. The fact that it permits data modelling and control flow constructs to be embedded inside computational ones is an argument *against* your proposal, not one in its favour.

and make it much more expressive.

Allowing data modelling and control flow constructs to be embedded inside expressions doesn't make the language more expressive overall, it just lets you do more within a single expression without giving the results of suboperations names and/or factoring them out into separate usable functions. The number of cases where executing a suboperation first and binding the result to a name even arguably reduces clarity are relatively few and far between, and better represented in PEPs like 403 and 3150 than they are in a general purpose statement-as-expression syntax. So if you're interested in pursuing this further, I suggest focusing on the possible pragmatic benefits of defining a standard way to tunnel Python code through whitespace insensitive contexts and reformat it with a pretty printer at the far end, rather than the far more nebulous concept of expressiveness. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Wed, Jan 7, 2015 at 1:08 PM, Yawar Amin wrote:

But I think I see what's happening here: statements are the top dogs in Python, and when Python wants a statement but only has an expression, it 'promotes' the expression into a statement by evaluating it, throwing away its value, and pretending nothing happened (i.e., that there was a 'pass' statement there). Thus defining a class can have the nonsensical effect of calling a function.

One type of statement is (via a couple of levels of indirection) the expr_stmt, which contains an expression and (if I'm not misreading the grammar) may or may not assign it to anything. https://docs.python.org/3/reference/grammar.html It's not that an expression gets "promoted", it's that one valid form of statement is an expression with nothing around it. In C, the definition is more like "an expression followed by a semicolon", but in Python, you don't need any adornment, so it looks identical. However, there is (as far as I know) no way in Python to wrap a statement into an expression. Some languages have such a thing (eg a lambda function syntax, which is an expression returning a function, and in which statements are allowed), but I don't think Python does, anywhere; all compound units that allow statements are themselves statements. ChrisA

On Jan 6, 2015, at 21:08, Yawar Amin wrote:

Actually I can offer counter-examples to that:

def p(x): print x class A: p("Hello!") class B: 1 # Etc.

No, there are no counter-examples there. A top-level program is a sequence of statements. Both def and class are compound statements, meaning the colon is followed by either a simple statement list or an indented block of statements. An expression statement is a kind of simple statement containing nothing but an expression.

But I think I see what's happening here: statements are the top dogs in Python, and when Python wants a statement but only has an expression, it 'promotes' the expression into a statement by evaluating it, throwing away its value, and pretending nothing happened (i.e., that there was a 'pass' statement there). Thus defining a class can have the nonsensical effect of calling a function.

But there's currently no way of going in the other direction, i.e. demoting a statement to an expression. Which is what I was trying to do.

That's the key. A function is a way to wrap a sequence of statements (the function body) so it can be used in an expression (a function call). Because JavaScript lets you define functions (with full statement syntax) in expressions, that gives you a way to "demote" an expression inline, while Python can only do so out-of-line. At the core, this is what all of the multiline lambda attempts and similar proposals are trying to accomplish.

On 2015-01-07 11:20, Yawar Amin wrote:

On 2015-01-07, at 2:11, Andrew Barnert wrote:

...

[...]

...

That's the key. A function is a way to wrap a sequence of statements (the function body) so it can be used in an expression (a function call). Because JavaScript lets you define functions (with full statement syntax) in expressions, that gives you a way to "demote" an expression inline, while Python can only do so out-of-line. At the core, this is what all of the multiline lambda attempts and similar proposals are trying to accomplish.

Thanks. This has given me an idea that's on a slightly different tangent than this one. Will write it up (this time with some actual runnable code) when I get home tonight.

As promised, a write-up of my idea: http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html I have a feeling that some Pythoneers will really hate it, but hey, what can you do :-) Regards, Yawar

On 2015-01-07 20:12, Chris Angelico wrote:

[...] If you're coming to python-ideas, you probably should have in your head that 'print' is a function and not a statement :)

Ah, I'll have to remember that from next time onwards to continue hiding my shame: I'm still on 2.7.3.

I'm really not seeing the advantage of this system; instead of expressing your code as a series of statements, you have to nest everything inside everything else, and the advantage is... the purity of having everything be an expression. Seems to me this has purpose in penetration testing (eg proving that you can get past someone else's sandboxing), but not practical programming.

Agreed, it doesn't look too great, but then again every idea starts out with someone saying 'That's never going to have a practical application', so I could take that as a good sign :-) Regards, Yawar

Hi Ben, On 2015-01-07 20:47, Ben Finney wrote:

[...] If you are interested in the future of Python (which is what this forum entails), you will be at cross purposes until you switch to Python 3.

Those who want to continue writing for Python 2 are welcome to do so, but Python 2 is not a consideration for discussing improvements to Python.

I'm aware of the current Python version and, as far as I can tell, have never proposed anything that would only work with an older version. The comments you're replying to are about something I posted on my own blog, which you'll agree is not python-ideas. Regards, Yawar

On Jan 7, 2015, at 17:05, Yawar Amin wrote:

On 2015-01-07 11:20, Yawar Amin wrote:

...

On 2015-01-07, at 2:11, Andrew Barnert wrote:

...

[...]

...

...

That's the key. A function is a way to wrap a sequence of statements (the function body) so it can be used in an expression (a function call). Because JavaScript lets you define functions (with full statement syntax) in expressions, that gives you a way to "demote" an expression inline, while Python can only do so out-of-line. At the core, this is what all of the multiline lambda attempts and similar proposals are trying to accomplish.

Thanks. This has given me an idea that's on a slightly different tangent than this one. Will write it up (this time with some actual runnable code) when I get home tonight.

As promised, a write-up of my idea: http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html

I have a feeling that some Pythoneers will really hate it, but hey, what can you do :-)

Well, I think you've successfully proven that for imperative-style code, a statement-expression language is more readable than a CPS pure-expression language. :) The difference isn't just a lack of syntactic sugar: in a pure expression language, "is followed by" has to be modeled as "is controlled by" (with CPS that's made explicit), meaning (in Python) it has to be indented. Haskell's do statement has a nice way out of that (if you really want to turn off Pythonistas by using the word "monad"). Promise-based sequencing (as with most JS Promises/A-based code) offers a different solution with a similar effect. It might be worth following one of those ideas to see where it leads you. In fact, the latter raises another possibility: once you're yielding from promises, you've got coroutines for free. In at some cases, explicit CPS code can be turned into implicit coroutine code very easily. If that works for everything, it also means you no longer need the manual trampoline and/or you can make all of your sequencing asynchronous and therefore add in concurrency for free (as in asyncio). Another trick that might be worth playing with explicit native globals and locals for your functions (e.g., by constructing a new types.FunctionType), so let bindings (which also take care of def, by just binding a lambda, and class, by binding a call to type or the appropriate metaclass) can use native Python environments (and closures). This would be more interesting if you wanted to add state later (because you'd get it for free), but it might be interesting just to see where you run into problems. Anyway, while this raises some interesting ideas, I don't think it raises any ideas for the future of the Python language, so I'm not sure Python-ideas is the right place to discuss it.

On 8 January 2015 at 11:05, Yawar Amin wrote:

As promised, a write-up of my idea: http://yawar.blogspot.ca/2015/01/expressive-functional-programming-with.html

The statement/expression distinction is deliberate, for the reasons you describe in the final article of your post. You will have zero chance of persuading anyone if you start from an assumption that the distinction is accidental. That said, there are some *very* limited situations where pulling things out-of-line in order to name them reduces readability (see PEPs 403 and 3150 for further discussion of that), and other situations where semantically significant leading whitespace causes practical problems (see http://python-notes.curiousefficiency.org/en/latest/pep_ideas/suite_expr.htm... for the discussion of that). If you can come up with a practical proposal that better handles those situations, *without* having a generally negative effect on code readability, and without introducing "two ways to do it" for various problems that will complicate learning, then you may have something signficant. But for something as fundamental as introducing new syntax, you need to start with concrete use cases, rather than hypothetical "it might be useful if..." situations. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 9 January 2015 at 18:46, Andrew Barnert wrote:

On Thursday, January 8, 2015 11:35 PM, Nick Coghlan wrote:

...

The statement/expression distinction is deliberate, for the reasons you describe in the final article of your post. You will have zero chance of persuading anyone if you start from an assumption that the distinction is accidental.

It strikes me that it might be useful if there were some good official explanation of this (maybe in the Design & History FAQ).

Many other "modern" languages (Ruby, CoffeeScript, etc.) try to eliminate or hide the distinction as much as possible. And JavaScript documentation often apologizes for having the distinction and explains how you can "work around the problem" by defining and calling a function inline to wrap any statement. So, many people believe that the distinction is all negatives with no positives.

I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.

But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.

Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.

For myself, the analogy I try drawing these days is to suggest to people that they look at a cookbook written in English, and then try translating that cookbook into their preferred programming language in such a way that anyone who could read the original English version could still read the translated version without needing to be told "Oh, don't worry about that bit, it's just needed so the computer understands what is going on"). When a problem is amenable to being handled in that style, you're often best off coding it in that style, as that opens up future maintenance to the broadest possible base of contributors. Languages that support coding in this style are the ones typically referred to as "scripting languages", and Python is one of those languages. However, there are additional styles of thinking that folks can be trained in (like lambda calculus, object oriented programming, event driven programming, etc) and the reason those styles of thinking were created is because the cookbook style starts to encounter scalability problems when trying to handle higher levels of complexity. Many programming languages are written on the assumption that a single style of thinking should be applied to the entire problem space of an application, and then get referred to by the style of thinking they promote (so functional programming languages, object-oriented programming languages, logical assertion languages, etc). This generally limits the applicability of those languages outside the problem domains where that particular style of thinking is suitable, and also means their barriers to entry are much higher than those of the scripting languages (since you need to learn a new way of thinking first, whereas scripting languages often let you talk to the computer using the same kind of reasoning that you'd use to give detailed step-by-step instructions to another human). The design of Python takes a more pragmatic approach and recognises that different problems and situations are best handled through different kinds of thinking, and so it offers those features as optional add-ons to the cookbook approach, allowing people to use them if they make sense in their particular case, *without* making them the dominant mode of operation of the overall language. Where folks run into trouble with Python (especially when it comes to the design side of things) is in trying to ignore that overarching cookbook element, and *just* use one of the other styles of thinking. It's not designed to work that way - the "cookbook" layer defines the overall structure, and then the other aspects like the object-oriented programming support, the functional programming support and the event driven programming support all reside within that as suitable tools for building *components* of larger systems. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Jan 20, 2015, at 9:15, "Franklin? Lee" wrote:

On Fri, Jan 9, 2015 at 3:46 AM, Andrew Barnert wrote:

...

I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.

But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.

Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.

Sorry to be late.

Will this Guido post suffice as a substitute? http://www.artima.com/weblogs/viewpost.jsp?thread=147358

That's not what I was looking for--but it is a great counter to my dismissal of "just to keep the parser simpler". Guido is of course right. I wasn't thinking about the fact that statement parsing and expression parsing are different modes, with state, and it's a lot harder for a human mind to subconsciously keep a stack of stateful modes in short-term memory than for a program. But I still think Python gains more from the inescapable statement-expression than this. Guido's answer is sufficient in itself, but there's more there even if it weren't true. I suppose the way to answer that would be to come up with a language where complex expressions are also indentation-sensitive (and there's no paren or backslash continuation) and show that if you get rid of the stack of stateful modes, there really is no other problem putting statements inside expressions. That's kind of the opposite of the way people usually try to tackle this problem, but if Guido's right that the usual way is inherently bound to fail...

Yawar might also need this, if he chooses to pursue this path. https://wiki.python.org/moin/AlternateLambdaSyntax

On Tuesday, January 20, 2015 3:23 PM, Guido van Rossum wrote:

On Tue, Jan 20, 2015 at 2:57 PM, Andrew Barnert wrote:

On Jan 20, 2015, at 9:15, "Franklin? Lee" wrote:

...

...

On Fri, Jan 9, 2015 at 3:46 AM, Andrew Barnert wrote:

...

I believe the distinction isn't just there to keep the parser simpler, or because Guido wasn't clever enough to figure out how to embed statements in expressions in an indentation-significant language, or because the examples of Ruby and CoffeeScript didn't exist yet and now it's too late for Python; statements are a big part of what makes Python code more readable than those languages, both in-depth and in a quick scan to see the general flow control and state changes.

But it's hard to explain why. And, even if I manage to explain why _I_ think statements make Python more readable, that's just my opinion; it would be nice to have something that Guido endorsed.

Also, for people who want to suggest changes to Python (or design their own languages), it would be helpful to understand how the distinction fits in with other things. For example, in a pure functional language, there may be no benefit to having statements. In a language where (almost) everything is an expression, having mutating methods return self instead of None wouldn't be nearly as much of a problem. More generally, expression-ness and not-mutating are deeply connected (e.g., comprehensions aren't designed for mutation, loop statements are), but it's not easy to say how. And so on.

Sorry to be late.

Will this Guido post suffice as a substitute? http://www.artima.com/weblogs/viewpost.jsp?thread=147358

That's not what I was looking for--but it is a great counter to my dismissal of "just to keep the parser simpler". Guido is of course right. I wasn't thinking about the fact that statement parsing and expression parsing are different modes, with state, and it's a lot harder for a human mind to subconsciously keep a stack of stateful modes in short-term memory than for a program.

But I still think Python gains more from the inescapable statement-expression than this. Guido's answer is sufficient in itself, but there's more there even if it weren't true.

I suppose the way to answer that would be to come up with a language where complex expressions are also indentation-sensitive (and there's no paren or backslash continuation) and show that if you get rid of the stack of stateful modes, there really is no other problem putting statements inside expressions. That's kind of the opposite of the way people usually try to tackle this problem, but if Guido's right that the usual way is inherently bound to fail...

I've lost track which side you are now arguing for, but one example of a language that takes indentation in complex expressions to the extreme is Coffeescript. Having had to reverse-engineer a fairly complex piece of Coffeescript recently, I definitely thing they went too far. It seems there is no formal grammar for Coffeescript, just a translator, and even if there were a formal grammar, it would have many odd corners and sharp edges.

I'm arguing the same thing: anyone who wants to embed statements inside expressions needs to understand why statements and expressions are distinct in the first place. Your old post that Franklin linked explains, given that we have that distinction, and significant indentation for statements, why any attempt to embed statements inside expressions is going to be inherently complex, and therefore likely a bad idea. But what I was hoping for was a post where you explained why we have that distinction in the first place. If Ruby and CoffeeScript don't need statements, why does Python? I think there's an answer, and it's part of the reason why code in Python tends to be more readable than in those languages (although, as you pointed out, CoffeeScript has other tangential problems), but I was hoping for _your_ answer.

On 21 January 2015 at 11:16, Guido van Rossum wrote:

So it's hard to say to what extent this was a conscious choice and to what extent it was just tradition. But there's nothing necessarily wrong with tradition (up to a point). I think it still makes sense that statements are laid out vertically while expressions are laid out horizontally. Come to think of it, mathematics uses a similar convention -- a formula is laid out (primarily) horizontally, while a sequence of formulas (like a proof or a set of axioms) is laid out vertically.

I really like that "each statement is like a single step in a mathematical proof" analogy. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Friday, January 23, 2015 2:06 PM, Ron Adam wrote:

...

...

On 21 January 2015 at 11:16, Guido van Rossum wrote:

...

...

So it's hard to say to what extent this was a conscious choice and to what extent it was just tradition. But there's nothing necessarily wrong with tradition (up to a point). I think it still makes sense that statements are laid out vertically while expressions are laid out horizontally. Come to think of it, mathematics uses a similar convention -- a formula is laid out (primarily) horizontally, while a sequence of formulas (like a

On 01/23/2015 08:56 AM, Nick Coghlan wrote: proof or a

...

...

...

set of axioms) is laid out vertically.

...

I really like that "each statement is like a single step in a mathematical proof" analogy.

I agree.

There are some other distinctions. When you consider these, you can see how many language designers come to similar solutions.

Expressions evaluate in unique name spaces, while statements generally do not. Consider "a + b"; it is evaluated in a private method after the values a and b are passed to it.

I don't think that's true. Consider "a[b] = c", which is a statement, but it's evaluated in a private method after the values a, b, and c are passed to it. The fact that it's a.__setitem__ rather than a.__add__ doesn't seem particularly important. I think the key is that __setitem__ doesn't have a return value--like (nearly) everything in Python that mutates state--and therefore there's no expression for it. The question is, what does that buy?

Statements are used to mutate the current name space, while expressions generally do not.

This is closer. I think, more generally, statements are used to mutate the important state that the local function is all about mutating. And the thing that gets mutated is almost always the leftmost thing. That definitely helps in scannability.

Statements can alter control flow, while expressions generally do not.

Sure, but I think this part only really helps if control flow is somehow visible. In Python, it is, because control flow almost always means compound statements, which means indentation, and very little else means indentation.

Having a clear distinction between expressions and statements makes reading and understanding code much easier.

Definitely. That's the part I think is key, but am struggling to explain. I think Guido offers a great analogy in mathematical proofs, but the question is to find the actual commonality behind the analogy. After some more thought on this, I think what it comes down to is that (idiomatically-written) Python lets you skim the control flow (because all non-trivial control flow, and very little else, is expressed in terms of indentation) and the state transitions (because each statement generally mutates at most one thing, and it's the leftmost thing), so you can quickly find the part of the code you actually need to read carefully, instead of having to read the whole thing. I've written this idea up in a bit more detail here: http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.htm...

I believe Python follows most of these conventions in most places, and when it doesn't, it's usually for a practical reason that are fairly obvious.

For example, an "or" expression is a bit of both.

Another example of how python chooses a practical alternative is we can nest expressions instead of using "call" statements on separate lines and a stack to hold the augments and return values. That is what python does in the byte code so we don't have to do it in explicit statements.

If you factor out all expressions you get byte code. Or if you factor out all statements you get something like lisp.

I don't think either part of that is true. Bytecode is full of things that are expressions—even your paradigm case of an operator expression is handled by an opcode. And conversely, CoffeeScript (if you avoid break/continue/return statements) factors out all statements, and Ruby comes close to doing so, and yet they're really not more Lisp-like than Python in any meaningful way.

Of course this subject is definitely a very subjective one which relays on agreeing on the general meaning of the above sentences. OR... YMMV.

On Jan 23, 2015, at 23:53, Ron Adam wrote: I fear we're probably getting pretty far off-topic for python-ideas, but I'm going to respond anyway.

On 01/24/2015 12:06 AM, Andrew Barnert wrote:

...

On Friday, January 23, 2015 2:06 PM, Ron Adam wrote:

...

...

On 01/23/2015 08:56 AM, Nick Coghlan wrote:

...

...

...

...

...

I really like that "each statement is like a single step in a mathematical proof" analogy.

I agree.

There are some other distinctions. When you consider these, you can see how many language designers come to similar solutions.

Expressions evaluate in unique name spaces, while statements generally do not. Consider "a + b"; it is evaluated in a private method after the values a and b are passed to it.

...

I don't think that's true.

Not all the time, which is why I said "generally". :-)

...

Consider "a[b] = c", which is a statement, but it's evaluated in a private method after the values a, b, and c are passed to it. The fact that it's a.__setitem__ rather than a.__add__ doesn't seem particularly important. I think the key is that __setitem__ doesn't have a return value--like (nearly) everything in Python that mutates state--and therefore there's no expression for it. The question is, what does that buy?

Right, and good question, but consider that it's not adding or altering the name space the 'a' object is in. It's a convenience/exception for using objects.

I think that's the point. In theory, objects are just syntactic sugar for closures; in practice, objects are an intuitively useful way to represent mutable state. And the fact that things that mutate an object are statements (whether directly, because they're __setattr__ type calls, or indirectly, because they're expressions that return None and can therefore only be used in expression statements) is important in making Python readable in practice.

If objects were done with closures, then the same modification would be done with an assignment statement. And then the generalisation would be more consistent. But at a cost is other ways.

...

...

...

Statements are used to mutate the current name space, while expressions generally do not. This is closer. I think, more generally, statements are used to mutate the important state that the local function is all about mutating. And the thing that gets mutated is almost always the leftmost thing. That definitely helps in scannability.

I'm not quite sure I follow that. I think what you are calling the important state, I think of as the shared state. A value that will be used multiple times in the same frame.

By "important state", I just mean whatever state the reader is likely to care about. And such state is almost always modified by a statement with some readable identification on the left side--whether it's a global/closure/local assignment, an attribute assignment, an element assignment, and augmented version of any of the above, a method call that doesn't return self (and will therefore only be used in an expression statement). So each statement means (at most) one state transition. And it's almost always the leftmost thing that's affected. And I think that aids readability.

Binding a name to a value mutates the names pace, but it does not mutate the name. It's still the same name.

Again this can be view multiple way if you consider how it actually works. Some languages use a stack to implement a name. And in a new frame, a new bound value would get pushed on the name stack. I'm not completely sure that python doesn't do something like that in some places to speed thing up. But I don't think so.

Sure, pure non-mutating languages can use a stack to implement bindings. But in a mutating language, that doesn't work if you have closures. CTM explains what you get out of mutable state (and what it costs) very nicely. Of course in practice any Python implementation has to be able to detect whether a given scope might have closures referring to it, so it _could_ switch to rebinding using a bindinmg stack, but in practice any Python implementation is likely to convert local accesses to offsets as CPython does, which is a much better optimization (at least for a language where rebinding is idiomatically common) that precludes that option.

...

...

...

Statements can alter control flow, while expressions generally do not. Sure, but I think this part only really helps if control flow is somehow visible. In Python, it is, because control flow almost always means compound statements, which means indentation, and very little else means indentation.

You are referring to the visual aspect, while I'm referring to what a statement does. Same thing. ;-)

Well, I'm highlighting the visual aspect because I think that's important to Python's readability, and to why statements contribute to that readability. If statements and expressions had similar indentation rules (as in CoffeeScript), I don't think Python would get the same benefit from having statements.

...

...

...

Having a clear distinction between expressions and statements makes reading and understanding code much easier. Definitely. That's the part I think is key, but am struggling to explain.

I think Guido offers a great analogy in mathematical proofs, but the question is to find the actual commonality behind the analogy.

Form follows function... Just one way to look at it. Sometimes what something does can come from the shape it has too.

Yes. But sometimes the shape is limiting, rather than expanding--and yet that limitation itself can be used to add meaning. (See Guido's point about code that fits in a window/screen/page.) So function partly follows form. And mathematical proofs are a great example. The fact that there are a limited number of ways you're allowed to get from the previous statements to the next one makes each step more readable.

...

After some more thought on this, I think what it comes down to is that (idiomatically-written) Python lets you skim the control flow (because all non-trivial control flow, and very little else, is expressed in terms of indentation) and the state transitions (because each statement generally mutates at most one thing, and it's the leftmost thing), so you can quickly find the part of the code you actually need to read carefully, instead of having to read the whole thing.

I've written this idea up in a bit more detail here:

http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.htm...

Very interesting.. and thanks for the mention. ;-)

Sure; as I said, your last paragraph (well, the last one I quoted) puts the whole thing I'm trying to answer much more clearly than I've been able to.

One of the things that makes a difference is to be able to hold a simplified model in your mind while you are working on it.

Yes! That's something else that was on the tip of my tongue that I couldn't explain clearly. Being able to hold enough of the syntax in your head to parse code subconsciously (which CoffeeScript lacks, as Guido pointed out) is part of it, but you're right, the big deal is being able to hold the entire model in your head. And at a different level, that's what makes scanability of flow control and state changes so important, which I couldn't explain before. That's the model of an imperative-style function that you need to be able to internalize to understand the function holistically. Thanks.

The separation of statements and expressions definitely helps me with that. If I can easily keep the name space in my mind... or at least the part of it that correspond with the block of code I'm looking at, It really helps me to visualise things in a meaningful way and see what effect the statements will have on the name space.

Exactly. Which is why the "one obvious mutation per line (usually)" property is so important. But I think describing it purely in terms of the local namespace hides the fact that it applies just as well to OO-style code (where most mutation is to the namespace of self or one of the other parameters) as to traditional structured imperative code.

When statements and expressions don't represent what they do in an obvious way, then all bets are off. It becomes a mental stumbling block.

Much of this became clear to me when I wrote a simplified script language that adds statements to a mini scheme like language. I did need to use braces for blocks. Even though I didn't need to use visual indentation, I still preferred formatting the programs in a similar style to python. In this mini language, the separation of statements and expressions is even more pronounced because all expressions are s-expressions, and all statements are not expressions.

And going one bit further, I used dictionaries for names spaces, like python, but used lists for statement blocks. So statement blocks have order, but name spaces don't.

...

...

...

I believe Python follows most of these conventions in most places, and when it doesn't, it's usually for a practical reason that are fairly obvious.

For example, an "or" expression is a bit of both.

Another example of how python chooses a practical alternative is we can nest expressions instead of using "call" statements on separate lines and a stack to hold the augments and return values. That is what python does in the byte code so we don't have to do it in explicit statements.

If you factor out all expressions you get byte code. Or if you factor out all statements you get something like lisp. I don't think either part of that is true.

It wasn't meant to be taken absolutely literally. Which is why I said... "something like".

OK, but I don't think it's figuratively true in any useful sense either. Languages like Ruby prove that removing statements doesn't have to mean something like Lisp. We aren't stuck with the models of the 60s. So the question of what you give up by factoring out all statements turns out to be more complicated (and more interesting) than it was in those models.

There is quite a bit of wiggle room when it comes to how different people think about things, and what words we use to describe them. Some times the hardest part of a discussion is getting the same mental picture in more than one person. :-)

...

Bytecode is full of things that are expressions—even your paradigm case of an operator expression is handled by an opcode.

I see opcodes as being similar to keywords that are used in a statement form. And I view the stack as being a temporary value space. Of course, I know certain combinations of several opcodes together may correspond to a particular python expression, but individually, each byecode along with it's following few values, are byte code statements to me.

And even if a single bytecode was the equivalent of a python expression, my point was you can use statements replace statements with expressions.

I don't know what you meant here. Maybe that a stack machine language has a fixed, non-extensible set of expressions, but its set of statements can be effectively arbitrarily extended with jsr/ret to other bytecode? If so, I'll buy that, but I'm not sure how it's relevant. I don't know of any examples of readable expression-free code that compare to such examples of readable statement-free code as, say, anything written in OCaml, or any Ruby code that sticks to single-return style. Eliminating expressions is clearly a non-starter for a readable language; eliminating statements is actually arguable--and the whole point is to find the arguments against doing so.

Byte code is a good example of that. But you still have functions calls.. You just can't nest them in the same way you do with python function calls, you must push them on the stack and use CALL "statements" to execute it, and then use another statement to store the value that gets put on the top of the stack. (or push another function on the stack that will use that value...)

...

And conversely, CoffeeScript (if you avoid break/continue/return statements) factors out all statements, and Ruby comes close to doing so, and yet they're really not more Lisp-like than Python in any meaningful way.

See the next paragraph... ;-)

...

...

...

Of course this subject is definitely a very subjective one which relays on agreeing on the general meaning of the above sentences. OR... YMMV.

Sure, but I'm not sure I understand the meaning you're going for. And, more importantly, I think there is an objective sense in which Python uses statements to gain subjective readability, and that objective sense is something we can question and try to answer.

On 25 Jan 2015 03:34, "Guido van Rossum" wrote:

Can you guys get a room?

More explicitly - I think we've established we *don't actually know* why the statement/expression distinction improves readability, just that decades of experience suggests it does. Further speculation here is unlikely to bring further clarity, so anyone interested in seeing it pursued to the point of reaching a more definitive conclusion would be well advised to get in touch with folks that actually research programming language readability (e.g. the authors of this paper from a few years ago: http://neverworkintheory.org/2011/10/24/an-empirical-comparison-of-the-accur... and their more recent follow-up http://neverworkintheory.org/2014/01/29/stefik-siebert-syntax.html) Cheers, Nick.

On Sat, Jan 24, 2015 at 3:06 AM, Andrew Barnert

...

On Jan 23, 2015, at 23:53, Ron Adam wrote:

I fear we're probably getting pretty far off-topic for python-ideas, but

I'm going to respond anyway.

...

...

On 01/24/2015 12:06 AM, Andrew Barnert wrote:

...

On Friday, January 23, 2015 2:06 PM, Ron Adam wrote:

...

...

On 01/23/2015 08:56 AM, Nick Coghlan wrote:

...

...

...

>> I really like that "each statement is like a single step in a >> mathematical proof" analogy.

I agree.

There are some other distinctions. When you consider these, you can see how many language designers come to similar solutions.

Expressions evaluate in unique name spaces, while statements generally do not. Consider "a + b"; it is evaluated in a private method after the values a and b are passed to it.

...

I don't think that's true.

Not all the time, which is why I said "generally". :-)

...

Consider "a[b] = c", which is a statement, but it's evaluated in a private method after the values a, b, and c

are

...

...

...

passed to it. The fact that it's a.__setitem__ rather than a.__add__ doesn't seem particularly important. I think the key is that __setitem__ doesn't have a return value--like (nearly) everything in Python that mutates state--and therefore there's no expression for it. The question is, what does that buy?

Right, and good question, but consider that it's not adding or altering the name space the 'a' object is in. It's a convenience/exception for using objects.

I think that's the point. In theory, objects are just syntactic sugar for closures; in practice, objects are an intuitively useful way to represent mutable state. And the fact that things that mutate an object are statements (whether directly, because they're __setattr__ type calls, or indirectly, because they're expressions that return None and can therefore only be used in expression statements) is important in making Python readable in practice.

...

If objects were done with closures, then the same modification would be done with an assignment statement. And then the generalisation would be more consistent. But at a cost is other ways.

...

...

...

Statements are used to mutate the current name space, while expressions generally do not. This is closer. I think, more generally, statements are used to mutate the important state that the local function is all about mutating. And the thing that gets mutated is almost always the leftmost thing. That definitely helps in scannability.

I'm not quite sure I follow that. I think what you are calling the important state, I think of as the shared state. A value that will be used multiple times in the same frame.

By "important state", I just mean whatever state the reader is likely to care about. And such state is almost always modified by a statement with some readable identification on the left side--whether it's a global/closure/local assignment, an attribute assignment, an element assignment, and augmented version of any of the above, a method call that doesn't return self (and will therefore only be used in an expression statement). So each statement means (at most) one state transition. And it's almost always the leftmost thing that's affected. And I think that aids readability.

...

Binding a name to a value mutates the names pace, but it does not mutate the name. It's still the same name.

Again this can be view multiple way if you consider how it actually works. Some languages use a stack to implement a name. And in a new frame, a new bound value would get pushed on the name stack. I'm not completely sure that python doesn't do something like that in some places to speed

...

Sure, pure non-mutating languages can use a stack to implement bindings.

But in a mutating language, that doesn't work if you have closures. CTM explains what you get out of mutable state (and what it costs) very nicely. Of course in practice any Python implementation has to be able to detect whether a given scope might have closures referring to it, so it _could_ switch to rebinding using a bindinmg stack, but in practice any Python implementation is likely to convert local accesses to offsets as CPython does, which is a much better optimization (at least for a language where rebinding is idiomatically common) that precludes that option.

...

...

...

...

...

Statements can alter control flow, while expressions generally do not. Sure, but I think this part only really helps if control flow is

somehow

...

...

...

visible. In Python, it is, because control flow almost always means compound statements, which means indentation, and very little else means indentation.

You are referring to the visual aspect, while I'm referring to what a statement does. Same thing. ;-)

Well, I'm highlighting the visual aspect because I think that's important to Python's readability, and to why statements contribute to that readability. If statements and expressions had similar indentation rules (as in CoffeeScript), I don't think Python would get the same benefit from having statements.

...

...

...

...

Having a clear distinction between expressions and statements makes reading and understanding code much easier. Definitely. That's the part I think is key, but am struggling to explain.

I think Guido offers a great analogy in mathematical proofs, but the question is to find the actual commonality behind the analogy.

Form follows function... Just one way to look at it. Sometimes what something does can come from the shape it has too.

Yes. But sometimes the shape is limiting, rather than expanding--and yet

...

So function partly follows form. And mathematical proofs are a great example. The fact that there are a limited number of ways you're allowed to get from the previous statements to the next one makes each step more readable.

...

...

After some more thought on this, I think what it comes down to is that (idiomatically-written) Python lets you skim the control flow (because all non-trivial control flow, and very little else, is expressed in terms of indentation) and the state transitions (because each statement generally mutates at most one thing, and it's the leftmost thing), so you can quickly find the part of the code you actually need to read carefully, instead of having to read the whole thing.

I've written this idea up in a bit more detail here:

http://stupidpythonideas.blogspot.com/2015/01/statements-and-expressions.htm...

Very interesting.. and thanks for the mention. ;-)

Sure; as I said, your last paragraph (well, the last one I quoted) puts

...

...

One of the things that makes a difference is to be able to hold a

simplified model in your mind while you are working on it.

...

Yes! That's something else that was on the tip of my tongue that I

couldn't explain clearly. Being able to hold enough of the syntax in your

...

And at a different level, that's what makes scanability of flow control

and state changes so important, which I couldn't explain before. That's the model of an imperative-style function that you need to be able to internalize to understand the function holistically.

...

Thanks.

...

The separation of statements and expressions definitely helps me with

...

Exactly. Which is why the "one obvious mutation per line (usually)"

...

But I think describing it purely in terms of the local namespace hides

...

...

When statements and expressions don't represent what they do in an

obvious way, then all bets are off. It becomes a mental stumbling block.

...

...

Much of this became clear to me when I wrote a simplified script

language that adds statements to a mini scheme like language. I did need to use braces for blocks. Even though I didn't need to use visual indentation, I still preferred formatting the programs in a similar style to python. In this mini language, the separation of statements and expressions is even more pronounced because all expressions are s-expressions, and all statements are not expressions.

...

...

And going one bit further, I used dictionaries for names spaces, like

...

...

...

...

...

I believe Python follows most of these conventions in most places, and when it doesn't, it's usually for a practical reason that are fairly obvious.

For example, an "or" expression is a bit of both.

Another example of how python chooses a practical alternative is we can nest expressions instead of using "call" statements on separate lines and a stack to hold the augments and return values. That is what python does in the byte code so we don't have to do it in explicit statements.

If you factor out all expressions you get byte code. Or if you factor out all statements you get something like lisp. I don't think either part of that is true.

It wasn't meant to be taken absolutely literally. Which is why I

said... "something like".

OK, but I don't think it's figuratively true in any useful sense either. Languages like Ruby prove that removing statements doesn't have to mean something like Lisp. We aren't stuck with the models of the 60s. So the question of what you give up by factoring out all statements turns out to be more complicated (and more interesting) than it was in those models.

...

There is quite a bit of wiggle room when it comes to how different

...

...

...

Bytecode is full of things that are expressions—even your paradigm case of an operator expression is handled by an opcode.

I see opcodes as being similar to keywords that are used in a

statement form. And I view the stack as being a temporary value space. Of course, I know certain combinations of several opcodes together may correspond to a particular python expression, but individually, each byecode along with it's following few values, are byte code statements to me.

...

And even if a single bytecode was the equivalent of a python

expression, my point was you can use statements replace statements with expressions.

I don't know what you meant here. Maybe that a stack machine language has a fixed, non-extensible set of expressions, but its set of statements can be effectively arbitrarily extended with jsr/ret to other bytecode? If so, I'll buy that, but I'm not sure how it's relevant. I don't know of any examples of readable expression-free code that compare to such examples of readable statement-free code as, say, anything written in OCaml, or any Ruby code that sticks to single-return style. Eliminating expressions is clearly a non-starter for a readable language; eliminating statements is actually arguable--and the whole point is to find the arguments against doing so.

...

Byte code is a good example of that. But you still have functions calls.. You just can't nest them in the same way you do with python function calls, you must push them on the stack and use CALL "statements" to execute it, and then use another statement to store the value that gets

...

...

...

And conversely, CoffeeScript (if you avoid break/continue/return statements) factors out all statements, and Ruby comes close to doing so, and yet they're really not more Lisp-like

wrote: thing up. But I don't think so. that limitation itself can be used to add meaning. (See Guido's point about code that fits in a window/screen/page.) the whole thing I'm trying to answer much more clearly than I've been able to. head to parse code subconsciously (which CoffeeScript lacks, as Guido pointed out) is part of it, but you're right, the big deal is being able to hold the entire model in your head. that. If I can easily keep the name space in my mind... or at least the part of it that correspond with the block of code I'm looking at, It really helps me to visualise things in a meaningful way and see what effect the statements will have on the name space. property is so important. the fact that it applies just as well to OO-style code (where most mutation is to the namespace of self or one of the other parameters) as to traditional structured imperative code. python, but used lists for statement blocks. So statement blocks have order, but name spaces don't. people think about things, and what words we use to describe them. Some times the hardest part of a discussion is getting the same mental picture in more than one person. :-) put on the top of the stack. (or push another function on the stack that will use that value...) than

...

...

...

Python in any meaningful way.

See the next paragraph... ;-)

...

...

...

Of course this subject is definitely a very subjective one which relays on agreeing on the general meaning of the above sentences. OR... YMMV.

Sure, but I'm not sure I understand the meaning you're going for.

And, more importantly, I think there is an objective sense in which Python uses statements to gain subjective readability, and that objective sense is something we can question and try to answer. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- --Guido van Rossum (python.org/~guido)

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi Franklin. On 2015-01-20 12:15, Franklin? Lee wrote:

[...] Will this Guido post suffice as a substitute? http://www.artima.com/weblogs/viewpost.jsp?thread=147358

Language design is not just solving puzzles? I read it very closely before starting this thread :-) To be honest, after reading it I came away thinking that language design/API design/whatever technical design really _can_ be just solving a puzzle--if you encode unseability as one of the parameters of the puzzle.

Yawar might also need this, if he chooses to pursue this path. https://wiki.python.org/moin/AlternateLambdaSyntax

Thanks, I read that pretty closely too. From what I could tell, they are all rather unfeasible for the simple reason that Python doesn't allow statements inside of expressions (nor should it--I believe; it would not be a good fit for the current syntax). The closest I've seen to one of those alternate lamba syntaxes being implemented is in the Mochi language, which compiles down to Python 3 bytecode. Personally, I've gone in a different direction--Google my name + lambdak and you will find it :-) Regards, Yawar