On Mon, Jul 13, 2009 at 06:51:27AM -0400, Chris Perkins wrote:

var callback = function(result) { // 10 lines of code... }; make_ajax_call(url, callback);

vs.

make_ajax_call(url, function(){ // 10 lines of code... });

For me, the latter is unreadable (it's hard to see where is the inner block, and where is the outer; where are the make_ajax_call's arguments and where are the function's arguments and body) even if the function is 1-2 line(s) long; if the function's body is longer the style becomes completely unacceptable for my not so sharp eyes and brain. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

This seems to be a nice idea. +0.5 -panzi On 07/13/2009 01:40 PM, Jan Kaliszewski wrote:

Dear Pythonistas,

It's my first post in the list, so let me introduce myself. My name is Jan Kaliszewski, I am a composer (studied in Frederic Chopin Academy of Music in Warsaw) and a programmer (currently working in Record System company on Anakonda -- ERP system being programmed in Python).

Comming to the matter...

13-07-2009 Chris Perkins :

...

In other words, I think code that puts things in the opposite order from the way you think of them is harder to read.

Another example: if I'm thinking that what I want to do is "substitute some stuff in a string", then I want to start by typing/reading "re.sub(...)", and not "def some_made_up_name(...):"

re.sub(pat, &, s) do(m): # several lines of code

vs.

def temp(m): # several lines of code re.sub(pat, temp, s)

The latter relegates the key line of code to the very last line, making it harder to see at a glance what it does.

So yes, the point of this really is just to allow you to write code "in the right order".

I like the idea of '&' (or another character/word in that role) -- but maybe it'd be enough to extend an existing feature: decorators with it, and to add possibility to use that special character/word as a function name after 'def'?

E.g.:

################### # Networking

dfr = twisted.whatever(...)

# @dfr.addCallback(&) could be equivalent @dfr.addCallback def &(result): handle(result)

@dfr.addErrback def &(err): handle_err(err)

################### # GUI (for some hypothetical library)

@my_widget.add_button('Go').on('click', &) def &(evt): if evt.ctrl_key: do_something() else: do_other_stuff()

# or, if we want to keep the result:

@my_widget.add_button('Go').on('click', &) def button_clicked(evt): if evt.ctrl_key: do_something() else: do_other_stuff()

################### # Threads

@thread.start_new: def &(): do_some_work()

Best regards, Jan Kaliszewski

This idea seems to be almost identical to my thread from November, "Proposal for Ruby-style anonymous block functions (that don't kill the indention)": http://mail.python.org/pipermail/python-ideas/2008-November/002340.html The real difference is that your proposal uses & (which is a bad idea, since & already has a meaning: __and__) and in foo() do: BODY it implicitly injects the BODY function into the args of foo, which violates Explicit Is Better Than Implicit. I agree though that sometimes rather than doing def spam(): BODY result = foo(spam) you would rather do result = foo(spam*) *To Be Defined on the next line, OK! def spam(): BODY because it reads better. As I understand it, this was the reason the decorator syntax was invented. Basically, @classmethod def foo(): BODY reads better than def foo(): BODY foo = classmethod(foo) so decorators are just syntatic sugar to improve readability. So, I agree with the basic motivation of your proposal, but I think the specifics of it aren't as good as my old one (although, I *would* think that, wouldn't I?) and from experience, I expect the BDFL to come and give you -2 in a couple days if the thread keeps up… In terms of refining my old proposal, it strikes me that ! isn't used in Python currently. If we have to use a sigil of some sort, we could try: new_list = sorted(old_list, key=!keyfunc) def keyfunc(item): normalize data… or something to that effect with the general semantic being that ! means "here comes a variable name, I will define this variable on the next line, please wait for it!" 2-cents-ly-yrs, -- Carl Johnson

2009/7/14 Chris Perkins :

Failing that, it looks like our proposals will die the same not-so-slow death for lack of interest. Oh well.

OK, so I'm not accused of "lack of interest"... :-) First of all, I have never felt the need for a construct like this. So it adds nothing but complexity to the language for me. But assuming that in the future, I do find a need for this, or there's a large group of other users who would find this useful, please can you address the following: Your fn() do (...): stmt1 stmt2 is equivalent to @fn def _(...): stmt1 stmt2 Why is your proposal better (ignoring "I like it better", as subjective preferences other than Guido's don't get the language changed)? For the case of function calls with arguments, why is fn(a, b, c) do(...): stmt1 stmt2 better than @functools.partial(fn, a, b, c) def _(...): stmt1 stmt2 (note that I could make my version look a bit nicer by a "from functools import partial", so you're still not allowed to argue aesthetics :-))? That leaves your general cases where you stick an "&" somewhere in the call as a placeholder. Only a couple of your motivating examples used that form, and to be honest I didn't find them very convincing. By the time you're getting that complex, I really do think you're trying to cram too much into one statement (and I know that's a subjective statement, sorry :-)) The same applies to the "x = fn(...) do (...)" cases that you include in the example, but don't explicitly state in your specification. I don't see any objective reason to think your form is clearer - yes, "put the assignment, which is the main thing, up front" is an argument in your favour, but "don't put too many things together just to avoid thinking of a name" is an equally valid against. OK, I hope that helps. I'm still against the idea, but I hope this gives you some idea why - it's not simply indifference. A construct like this needs to bring something better than mere improved readability to the language, in my view. Paul.

2009/7/14 Chris Perkins :

For me, avoiding thinking of a name was never the primary motivation - putting things in the "right order" is. But I do believe that there are situations where the context that a block of code sits in tells far more about its purpose that a name ever could - in those circumstances, a forced function name can be useless at best, and distracting clutter at worst. For example:

y = sorted(x, key=&) do(item):    name = item.split('-')[1]    return name.upper()

I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.

That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me). By contrast. def uc_name(item): name = item.split('-')[1] return name.upper() y = sorted(x, key=uc_name) is a lot clearer to me. And it would be even better if I knew what was going on with the splitting of items on dashes, and taking the second bit, was all about. Then, I could give the function a more meaningful name. [[An admission here - when I first wrote that example, I spelled the function name differently in the 2 places. So my "see how much better my way is" example was wrong - in a way yours couldn't be! In my defence, the error would have come up as soon as I ran the code, and I could say that if I had thought of a better name it wouldn't have happened - but nevertheless. In the spirit of friendly debate, I'll give you that argument for your case free of charge :-)]] My point is that *technically* I can see what the 2-line block of code does - but without giving it a name, I don't have a clue what it's *for*. But I don't imagine we're ever going to agree on this. Let it stand that (without implying any criticism - people differ) your idea of readable code is confusing and difficult to follow for me. Hence my distrust of readability arguments :-)

...

A construct like this needs to bring something better than mere improved readability to the language, in my view.

Aha! You said "improved readability"! So I'm going to conveniently forget everything else you said and write your name in the "pro" column. :)

Good try, but you don't win that easily :-) I don't think this construct improves readability. Personally, I think it *harms* readability (although I concede that like anything, it could be used tastefully). What I was saying was that your *only* argument was readability, and I don't think that is enough of an argument in itself.

Seriously, I guess this is where we differ - I think that improved readability _is_ a sufficient goal unto itself.  The only question is whether other people (OK, one other person in particular) thinks it improves readability, and if so, then by how much.

It has to improve readability, uncontroversially, in enough cases to justify the cost in terms extra of language complexity (both for implementation, and definition/teaching). To demonstrate the benefits, a good place to look is often the standard library. If your proposed change can be demonstrated to improve the code shipped with Python in the stdlib, in a number of places, people will start to listen. Also, apart from the basic cost of any change (the fundamental barrier to entry, if you like) the cost of your change is higher because there are obscure corner cases that aren't easy to explain - witness questions here about interaction of this form with if statements, etc. This is a unique construct in that it's the only indentation-sensitive multi-line element of an expression. People don't have an intuition about how it "should" work, because there's nothing similar to extrapolate from. Ultimately, that's why this idea (in one form or another) has been around for so long and never been accepted - nobody has ever managed to come up with an idea that prompted the "oh, yes, obviously!" reaction in people needed to confirm that it's the "right" solution. I'd like to think that one day, someone will come up with the ideal answer which will finally address all these "extended lambda"/"code block" needs. But I don't think it's happened yet. Sorry :-) Paul.

On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore wrote:

2009/7/14 Chris Perkins :

...

y = sorted(x, key=&) do(item):    name = item.split('-')[1]    return name.upper()

I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.

That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me).

Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation: y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() ) There must be somewhere posted the reason this was rejected but my google-fu is failing me. George [1] http://boo.codehaus.org/Closures

On Tue, Jul 14, 2009 at 11:55 PM, Chris Rebert wrote:

On Tue, Jul 14, 2009 at 8:50 PM, George Sakkis wrote:

...

On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore wrote:

...

2009/7/14 Chris Perkins :

...

y = sorted(x, key=&) do(item):    name = item.split('-')[1]    return name.upper()

I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.

That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me).

Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:

y = sorted(x, key=def (item):                               name = item.split('-')[1]                               return name.upper()               )

There must be somewhere posted the reason this was rejected but my google-fu is failing me.

IIRC, it screws with the essential statement-expression dichotomy of Python: http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html

Thanks, this was it. The important part is Guido's quote "any solution that embeds an indentation-based block in the middle of an expression is unacceptable", so it's basically a design judgement, not an implementation constraint. George

George Sakkis wrote:

Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:

y = sorted(x, key=def (item):                               name = item.split('-')[1]                               return name.upper()               )

FWIW, I find that completely ugly and non-Python-esque (unpythonic being perhaps too strong of criticism). :-D Clearly this is an issue which has a lot of good arguments on both sides, and since the final metric has to be "readability of code" it's going to hard to make a knock-down argument for one side or the other. I'm fine with the status quo for now: in Guido's gut we trust. My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem. Just solving that one problem by itself seems unlikely to make it into the language, since it is pure "sugar" and Pythoneers have a well justified wariness toward anonymous functions (why not just name them and be done with it?). Perhaps someone should take a look at the original "with" proposal, or some of the various "do" and "block" proposals. One idea I think about sometimes is having a more convenient way to do something along the lines of metaclass.__prepare__ but for functions… -- Carl Johnson

Greg Ewing wrote: Nitpicking:

* It's not restricted to a single function argument:

 def foo(f, g) where:    def f(x):      ...    def g(y):      ...

Surely, you meant foo(f, g) where: def f(x): ... def g(y): ... Ending a statement that ends in a ":" normally with a "where" is too ambiguous and ought to be prohibited. I need to think about this proposal some more, but it would help solve the issue brought up on the list recently that [f(x) for f in fs if f(x)] is inefficient and [i for i in (f(x) for f in fs) if i] is ugly. new_list = [fx for f in fs if fx] where: fx = f(x) The "where" on the end of the clause does catch your eye and make you think "OK, be aware, something is undefined on this line but will be defined on the next." OTOH, in new_list = [i for i in xs for j in ys if fx] where: fx = f(x) or new_list = [i for i in [j for j in ys if fx?] if fx?] where: fx = f(x) Does the "where" get re-used with each loop of the inner list comprehension or just the outer one? What about total = sum(count for item in l) where: count = item.size if item.size > 0 else 1 It seems like for that case, we definitely want the "where" to apply every time the generator expression loops though, but… Is that feasible? Hmm. -- Carl Johnson

Hello, 15-07-2009, 09:55 Greg Ewing wrote:

One such extension might be a "where" block. Applied to the current problem:

foo(f) where: def f(x): ...

At first glance, +1 from me. 15-07-2009, 12:25 Paul Moore wrote:

inside the [...] - which clearly violates the "no indented block structure within expressions" design rule.

My initial instinct was that the original form violated that rule, too - but if you interpret the where block as a statement postfix, you're (almost) OK. The (almost) bit is because the where block has to be limited to simple (single-line) statements.

stmt where: a b c =>

a b c stmt

with the proviso that variables defined in a,b,c are only available within a, b, c and stmt.

I don't like the direction of limiting content of the block to set of singular simple statements. I suppose where-block should simply create a separate nested scope. So...

x = 1 y = x where: x = 2 print x

What does that print?

I tkink it should print 1 -- because, if where-block creates a separate nested scope, its local variables shouldn't leak. But there is another question: what is the value of y: 1 or 2? Or, more genarally: should be the line ended with 'where' included *in* that nested scope or not? I suppose it should, so the final value of y should be 2. Another example: a = 7 b = 8 x = 'X' z = 'zzzzz' foo(a, b, c("It's..."), x) where: # a = input() in Python 3.x :) a = raw_input(z) # -> zzzzz try: a = int(a) except ValueError: a = 0 def b(bar): 'very complex function' print(a, bar) else: def b(bar): 'also very complex function' print(bar, a, bar) def c(*args, **kwargs): 'also also very complex function' print(a, b) # -> 7 8 Best regards, -- Jan Kaliszewski

On Wed, Jul 15, 2009 at 4:59 AM, Carl Johnson < cmjohnson.mailinglist@gmail.com> wrote:

total = sum(count for item in l) where: count = item.size if item.size > 0 else 1

It seems like for that case, we definitely want the "where" to apply every time the generator expression loops though, but… Is that feasible?

Unlikely, but this is: total = sum(f(item) for item in lst) where: def f(item): return item.size if item.size > 0 else 1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com

On Wed, Jul 15, 2009 at 2:55 AM, Greg Ewing wrote:

One such extension might be a "where" block. Applied to the current problem:

foo(f) where: def f(x): ...

I like this proposal much more than any of the previous proposals. It has some of the flavor of Lisp's "let", but puts the important line first instead of at the end. In a nutshell, it says "define these symbols for this one line only". On the downside, for the common case of wanting to define a single function, the body of the function must be indented twice. I suggest the following grammar and meaning, which would be referred to by the assignment, yield, return, and expression statements: where_expression ::= expression_list "where" ":" suite | expression_list It evaluates as follows. [return|yield|x=] expression_list where: suite is roughly equivalent to: def where_expression(): suite return expression_list [return|yield|x=] expression_list() How about the following as additional syntactic sugar for the common one-function case? x = blah(f) where def f(item): body_of_f -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com/

i like defining things prior to running them. One issue, however, is namespace management. i'd like temporary names to be discarded automatically. where: def foo(x,y,z): return x+y+z do: bar = foo(1,2,3) print foo(1,2,3) // foo is undefined! print bar // 6 you could create a local namespace w/ just the 'where' block. where: x = 1 y = x print x // x is undefined! Perhaps we could bind the variables and pass them around. where my_locals: x = 1 y = 2 print my_locals() // {'x': 1, 'y': 2} print x // x is undefined! - Jae On Jul 15, 2009, at 12:55 AM, Greg Ewing wrote:

Carl Johnson wrote:

...

My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem.

One such extension might be a "where" block. Applied to the current problem:

foo(f) where: def f(x): ...

Benefits include:

* It's pretty much self-explanatory, as it builds on the existing syntax for defining functions, and "where" is used in a similar way in mathematics and some existing programming languages.

* You get to give the function a meaningful name if you want.

* It's not restricted to a single function argument:

def foo(f, g) where: def f(x): ... def g(y): ...

* It needn't be restricted to functions:

foo(x, y) where: x = a * 42 y = b / 17

Drawbacks include:

* Execution happens out of order, although that's an inherent feature of the original proposal as well. There are precedents in the language already, such as list comprehensions and conditional expressions.

* Allowing arbitrary statements in the block could possibly be considered detrimental to code readability. To mitigate that, the block contents could perhaps be restricted to binding constructs only.

-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas

or maybe better:

while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY

Another possible variant: while foo(x) and bar() # no colon where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY But the former has such an advantage that where-block content is indented deeper than loop body -- what reflects that it (where-block) creates nested scope (contrary to loop body). -- Jan Kaliszewski

Greg Ewing wrote:

Jan Kaliszewski wrote:

...

while foo(x) and bar() where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY

That's kind of clever, although it does require modifying the while-loop syntax, and if you're doing that, there's not a lot of gain over introducing a dedicated loop-and-a-half syntax.

Also, once we're allowed 'where' clauses on while statements, people are going to want them on other kinds of statements as well:

if a == b where: a = foo() b = blarg() then: ...

class Ham(Spam) where: class Spam: ... is: ...

def f(x = default) where: default = whatever as: ...

try: ... except eels where: eels = something_bad() do: ...

Where do we stop?

...

while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY

I think I liked the first version better. There's something jarring about the unindent half way through without any keyword to mark the boundary. Also the bare 'where' relating to the previous line suggests some weird grammatical gymnastics going on.

"do" might be a sufficiently generic word to cover all the cases. while ... where: ... do: ... if a == b where: ... do: ... etc.

2009/7/17 Gerald Britton :

Using "where" would have the advantage of avoiding a function call, though I suppose it wouldn't save (much) stack space since it needs a new scope anyway.  I see the "where" clause as a useful way to abstract a section of code to keep the high level in view, then break it down as you go.

You're assuming that the implementation of where is faster than a function call. Given that the defined semantics are in terms of defining then calling a temporary function, this may well be an invalid assumption. (And if it isn't, then can the tricks which get used to speed up the where cause please be used to speed up function calls in general, too - as that would be far more generally useful! :-)) Paul.

On Fri, 17 Jul 2009 11:25:15 pm Gerald Britton wrote:

Sometimes I use helper functions to make things easier to read. However, I don't like unnecessary function calls since they are expensive in Python compared to other languages.

That's only a problem if: (1) Your code actually is too slow; and (2) You have profiled your code and discovered that the biggest contributor to your code's slowness is the cost of calling functions, not the execution time of the functions. Unless both of these cases hold, then I suggest you are guilty of premature optimization. Possibly even pessimation -- given the complexities of CPU caches, memory, swap, etc, it's possible that your code could be *slower*, not faster, due to your "optimization". People's intuitions about what's fast and what's slow in Python code are often completely at odds with reality. Unless you actually have a problem with slow code that needs solving, my advice is to not worry about it. Use good algorithms, use tried and tested fast data structures, and worry about micro-optimizations when you *actually* need them. -- Steven D'Aprano

To my eyes, the first is harder to read than the second one. Of course I can do this:

f = self.someobject.get_generator() g = self.important_test mylist = [item in f if g(item)]:

but then "f" and "g" pollute the calling context's namespace.

So what? Seriously, what harm does a temporary association do? Especially when one can easily 'clean up' with 'del f,g' if it actually make a functional difference or if one just wants to cater to an obsession. In particular, why is this reason to *PERMANENTLYT pollute the language with an otherwise useless new feature? Unlike namespace associations, deleting language features is really difficult. Terry Jan Reedy

"Jan Kaliszewski" writes:

And what about that:

class SomeClass: ...

def summarize_proust(self, objects, args, method_name) "Summarize Proust in 15 seconds"

return map(action, objects, args, alts) where:

def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result

alts = map(weirdtrans, objects) where def weirdtrans(obj): # some transformations not very # useful in more general context

You then make an invalid comparison of this versus making ‘weirdtrans’ an unnecessary instance method. But the correct equivalent of your example above is to make another local function:: class SomeClass: # … def summarize_proust(self, objects, args, method_name): """ Summarize Proust in 15 seconds. """ def weirdtrans(obj): # some transformations not very # useful in more general context alts = map(weirdtrans, objects) def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result return map(action, objects, args, alts) That seems more straightforward to me. Certainly I don't find the one with ‘where’ any clearer. -- \ “Holy unrefillable prescriptions, Batman!” —Robin | `\ | _o__) | Ben Finney

On Sat, 18 Jul 2009 06:28:40 am Gerald Britton wrote:

I often time my code and often find that removing function calls "where" possible actually makes a measurable difference. This is especially the case with little getters and setters in class defs.

Perhaps you shouldn't be writing Java code with lots of getters and setters then *wink* Seriously, I've very glad to hear you're measuring before optimising, but a "measurable difference" is not necessarily a significant difference. Anyway, we're not here to discuss the pros and cons of optimization, so back to the top on hand:

Anyway, I like the "where" idea not because of real or imagined performance gains, but because of its cleanness when expressing problems.

I just don't see this cleanness. It seems to me that the "where" construct makes writing code easier than reading code. It is a top-down construct: first you come up with some abstract expression: element = w + x.y - f(z) where: and then you need to think about how to implement the construct: w = 2 x = thingy() z = 4 That roughly models the process of writing code in the first place, so I can see the attraction. But it doesn't model the process of *reading* code: "Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any of those names before, are they globals? Ah, there's a 'where' statement, better pop that expression into short-term memory while I read the block and find out what they are..." In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process: def parrot(): x = 1 y = 2 z = 3 result = x+y+z return result A complication: if the function you are reading relies on globals, including functions, then you may need to jump out of the function to discover what it is. But if you already know what the function does, or can infer it from the name, then it doesn't disrupt the sequential reading. But the where clause introduces *look-ahead* to the process: def parrot(): x = 1 result = x+y+z where: # look-ahead y = 2 z = 3 return result This is not a heavy burden if you are limited to a few simple names, but as soon as you re-use pre-existing names in the 'where' scope, or increase its complexity, readability suffers greatly. def harder(): w = 5 z = 2 result = w + x.y - f(z) where: w = 2 class K: y = 3 x = K() def f(a): return a**2 + a z = 4 return result Look-ahead is simply harder on the reader than reading sequentially. You can abuse any tool, write spaghetti code in any language, but some idioms encourage misuse, and in my opinion this is one of them.

A big use for me would be in list comprehensions. In one project I work on, I see things like:

for i in [item in self.someobject.get_generator() if self.important_test(item)]

and other really long object references.

I don't see anything objectionable in that. It's a tad on the long side, but not excessively.

which I would like to write:

mylist = [item in f if g(item)] where: f = self.someobject.get_generator() g = self.important_test

I don't think much of your naming convention. Surely g should be used for the generator, and f for the function, instead of the other way around?

To my eyes, the first is harder to read than the second one. Of course I can do this:

f = self.someobject.get_generator() g = self.important_test mylist = [item in f if g(item)]:

but then "f" and "g" pollute the calling context's namespace.

Holy cow! How many variable names do you have in a single function that you have to worry about that??? Honestly, I think this entire proposal is a pessimation: you're proposing to significantly complicate the language and negatively impact the readability of code in order to avoid polluting the namespace of a function? Of course, then people will start worrying about "polluting the namespace" of the where-block, and start doing this: result = x + y + z where: x = 1 y = 2 z = a*b - c where: a = 5 b = 6 c = d*e where: d = 3 e = 4 -- Steven D'Aprano

Steven D'Aprano wrote:

It seems to me that the "where" construct makes writing code easier than reading code. It is a top-down construct: first you come up with some abstract expression:

element = w + x.y - f(z) where:

and then you need to think about how to implement the construct:

   w = 2    x = thingy()    z = 4

That roughly models the process of writing code in the first place, so I can see the attraction. But it doesn't model the process of *reading* code:

"Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any of those names before, are they globals? Ah, there's a 'where' statement, better pop that expression into short-term memory while I read the block and find out what they are..."

In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:

This objection seems to be a very convincing one. At the very least, I think we should give up on the idea of having the "where" block use a different scope, since that seems to cause more problems than it solves. For me, the original reason why where clauses looked appealing is that I think there are some very limited cases in which the code reads better out of order. My canonical example for this is "new_list = sorted(old_list, key=...)" It just seems like sometimes you don't want to hear about the key func until after you know that you're going to sort a list. A simple "where" (or even "do … where" which might read a little better) statement would solve this problem by basically just letting you one line out of order. do new_list = sorted(old_list, key=key) where: def key(item): do_data_normalizing_stuff… I also think that most decorator definitions would look better with the return at the top rather than the bottom: def my_debug_decorator(f): do return inner where: def inner(*args, **kwargs): print(args, kwargs) return f(*args, **kwargs) Yes, this nests a level deeper than usual (flat is better than nested), but I think it's a lot less confusing than the usual form of decorators with the return at the end where your eye doesn't know where to rest. On the other hand, it's not that hard to make a decorator decorator to get rid of the weird return clause, and having any where statement at all invites the abuse of nested wheres: do x += y where: do y = z where: do z = x where: x = 2 print(x) # 4 Is the temptation for abuse here so strong that it will out weigh the potential for readability gains? Does this construct violate TOOWTDI? My answer is a definite… maybe. So for now, I think "where" is not ready for prime time. My 2 cents, -- Carl Johnson

Greg Ewing writes:

In a sense, the where-block is in part an attempt to extend the assignment statement so that the "right hand side" can span more than one statement.

That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda URL:http://www.python.org/dev/peps/pep-3099/) that multi-statement expressions are verboten. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but why does a forklift have to be so big if all it does | _o__) is lift forks?” —_Pinky and The Brain_ | Ben Finney

Ben Finney wrote:

That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda URL:http://www.python.org/dev/peps/pep-3099/) that multi-statement expressions are verboten.

But it's not a multi-statement expression. It's a statement with a suite, and it works entirely within the existing framework of the grammar. -- Greg

Guido van Rossum wrote:

The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-).

However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence.

I believe it's at the other 'end for stick' of development process and doesn't hurt 3.1 at all. It could appear at the earliest in 3.2 or 3.3 in __future__... but I think it doesn't mean we should freeze working on new ideas, even if they have chance to be released not earlier that in three or five years. Mike Meyer wrote:

but the BDFL said he sort of liked it, except for the name - and that we should be shaking out 3.X instead of extending the language. So, just to get it on record, how about a B&D version of the where statement? Let's call it the "with" clause.

But we already have 'with' in the language, doing completely different things. I propose 'using'. Regards, zuo -- Jan Kaliszewski

Guido van Rossum wrote:

On Sun, Jul 19, 2009 at 11:29 AM, Jan Kaliszewski wrote:

...

Guido van Rossum wrote:

...

The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-).

However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence. I believe it's at the other 'end for stick' of development process and doesn't hurt 3.1 at all.

Actually, adding too many new features post 3.1 *could* hurt 3.1 -- it could slow adoption because people might decide to wait for 3.2 which they expect to be even better. Plus the implementation of 3.2 features uses up energy that would be better spent elsewhere (the Python community has a chronic shortage of hands to help with menial task like porting 3rd party code to 3.1).

Another argument against new features is that the people who have the hardest time moving to 3.1 are those who are most bothered by change in general. If they view the language as "moving too fast" they might decide to move to a language that moves more slowly, or just stick with Python 2.4, which is nearly as bad.

A more cynical view (the like of which I've heard expressed about Perl 6) would be that python-ideas is where we keep those folks occupied who love to argue about new features but are unlikely to contribute anything... :-)

Or it's where the bikeshedding gets done; if an idea makes out of python-ideas then any critic of the new feature can be pointed to the relevant thread and told "already discussed, see!". :-)

[OT] Steven D'Aprano wrote:

I've often wondered whether coding would be simpler (at least for beginners) if we wrote assignment left-to-right like this:

2*x + 1 => y

instead of

y = 2*x + 1

That might be a nice experiment for a teaching language some day: does left-to-right assignment reduce or eliminate the confusion that beginners experience over assignment?

In fact, the more I think about it, the more I like the idea.

Oh yes... Hypertalk does that, only very verbosely:

put 2*x + 1 into y

And doesn't Cobol do something similar?

Chuck does it, and just with '=>' :) (see: http://chuck.cs.princeton.edu/ ) *j

Gerald Britton wrote:

About return, yield and their ilk in where clauses:

don't do it!

Seriously, a where clause should be restricted to providing definitions for the things in the statement for which the where clause is specified. Today, we can't do a return (with argument) in a generator function, which makes sense. A where clause should not allow returns or yields or in general actions that explicitly transfer control beyond the statement.

+1 -- Jan Kaliszewski

On Sat, 18 Jul 2009 15:33:59 +1200 Greg Ewing wrote:

Steven D'Aprano wrote:

...

But the where clause introduces *look-ahead* to the process:

def parrot(): x = 1 result = x+y+z where: # look-ahead y = 2 z = 3 return result

I think this example is too rarified to give much idea of how readable a where-statement might be.

I, on the other hand, am convinced that this example is dead on. By requiring the definition to be contextually close to the use, you'll encourage people to use shorter, less meaningful names. Or can you honestly say that you use names of the same length and carrying as much meaning for local variables as you do for globals, attributes and parameters?

In real life you wouldn't use arbitrary names like x, y, z. You'd choose something to make the expression meaningful in its own right, so you can get the gist of what's going on just from reading the first line.

What where you're pointing your pronouns! I know you don't speak for me, and I'm pretty sure you don't speak for Steven (but I don't either, and he may agree with you). I'd considered where to be promising, assuming that the problems it has could be worked through. But Steven's observations are deadly. Not that the original impulse wasn't good - trying to find things that scratch the itches that cause repeated "function definition in an expression" proposals is almost certainly worthwhile. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org

On Sat, 18 Jul 2009 10:45:44 pm Daniel Stutzbach wrote:

On Fri, Jul 17, 2009 at 7:39 PM, Steven D'Aprano wrote:

...

In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:

Normally, that's true, but that's not true when defining a sub-function. Consider the following:

def foo(items): def compute_sort_value(item): # compute value based on item's properties return value # a bunch of other code is here items.sort(compute_sort_value)

Above, the body of compute_sort_value appears long before it is executed.

Right. That's no different from this case: def bar(items): c_s_v = list( # comment goes here 'csv') # a bunch of other code is here items.extend(c_s_v) def is an executable statement which creates a function; list() is an executable function (technically a type) which creates an list. In your example, the inner function compute_sort_value is created when foo is executed. Python functions are first class objects, and so you can create functions and pass them to functions without necessarily calling the __call__ method. In my example, the inner list c_s_v is created when bar is executed. Lists are first class objects, and so you can create lists and pass them to functions without necessarily calling the __getitem__ method. In both functions, the objects compute_sort_value and c_s_v are created before they are used.

Now consider:

def foo(items): # a bunch of other code is here items.sort(key=mean) where: def compute_sort_value(item): return value

Presumably you meant to write: items.sort(key=compute_sort_value) where:

Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort().

No it doesn't. The function definition appears outside the call to sort(). For the function definition to appear inside the call to sort, you'd need to write something like: items.sort(key=lambda item: value) In this case, the lambda only occurs inside the call to sort() -- it's inaccessible to anything else (excluding trickery inside of sort() itself). By comparison, the compute_sort_value() function exists inside the where-block, and therefore is accessible to everything else inside that block: items.sort(key=compute_sort_value) where: def compute_sort_value(item): return value print compute_sort_value('testing testing 1 2 3') -- Steven D'Aprano

On Sun, 19 Jul 2009 06:11:57 am Gerald Britton wrote:

...

   items.sort(key=compute_sort_value) where:

...

Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort().

No it doesn't. The function definition appears outside the call to sort(). For the function definition to appear inside the call to sort, you'd need to write something like

Yes, it does appear within the call. The call is more than just the parameters in parentheses. It begins with "items.sort" and ends (in this case) with then end of the "where" clause

That's a definition of "within" that I'm not really happy with. And I think it's wrong. Consider the timing of calls: import time class MyList(list): def sort(self, *args, **kwargs): print "Call to sort() started at %s seconds" % time.time() super(MyList, self).sort(*args, **kwargs) print "Call to sort() ended at %s seconds" % time.time() items = MyList([-9, 7, -4, 2, -1, 0, 3]) items.sort(key=func) where: print "Enter the where-block at %s seconds" % time.time() time.sleep(555) def func(item): return abs(item) You need to create func before you can pass it as an argument to sort(), so I would expect the above to print something like: Enter the where-block at 1234560000.23 seconds Call to sort() started at 1234560555.24 seconds Call to sort() ended at 1234560555.25 seconds To my mind, to say that the contents of the where-block occur "within" the call to sort, it would imply that you get output like this: Call to sort() started at 1234560000.23 seconds Enter the where-block at 1234560000.24 seconds Call to sort() ended at 1234560555.25 seconds

...

   items.sort(key=lambda item: value)

The idea behind "where" clauses is not to replace lambdas. In fact, they are useful together:

items.sort(key=mysort) where: mysort = sort_key_function_factory( with, many, arguments)

If you're going to say that where and lambda are useful together, shouldn't your example actually show where and lambda together?

which to many is preferable over:

items.sort(key=lambda item:sort_key_function_factory(with, many, arguments))

That's equivalent to: def f(item): function = sort_key_function_factory(with, many, arguments) return function items.sort(key=f) which almost certainly is not what you what. I think what you meant was: items.sort(key=lambda item: sort_key_function_factory(with, many, arguments)(item) ) # note that you call the function returned by the factory which at least works, but it adds a pointless layer of indirection. What you should have is: items.sort(key=sort_key_function_factory(with, many, arguments)) which of course is simpler than your earlier example with the where-block. -- Steven D'Aprano

On Sat, Jul 18, 2009 at 9:08 PM, Steven D'Aprano wrote:

...

Yes, it does appear within the call. The call is more than just the parameters in parentheses. It begins with "items.sort" and ends (in this case) with then end of the "where" clause

That's a definition of "within" that I'm not really happy with. And I think it's wrong. Consider the timing of calls:

I think we're arguing semantics there. There are some reasonable definitions of "within" where the "where" statement isn't within the call, and other reasonable definitions of "within" where it after the call. You've already given examples of the later. Here's an example of the former: for i in range(5): x.append(some_function(i)) I think we can agree that the body of the "for" loop is within the "for" statement? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com

MRAB wrote:

"do" might be a sufficiently generic word to cover all the cases.

if a == b where: ... do: ...

It's probably okay for 'if', but to me it doesn't read right with 'def' or 'class'. But in any case, I'm -0.9 on allowing 'where' on arbitrary statements in the first place. Seems like feature creep on a massive scale to me. (Feature stampede?) -- Greg

On Wed, Jul 15, 2009 at 4:27 AM, Paul Moore wrote:

The important principle that George quoted:

2009/7/15 George Sakkis :

...

Thanks, this was it. The important part is Guido's quote "any solution that embeds an indentation-based block in the middle of an expression is unacceptable", so it's basically a design judgement, not an implementation constraint.

is key here - keep blocks and expressions distinct. Future proposals should keep that in mind.

I think you may be misinterpreting - note Guido's use of the word "middle". That is precisely why I proposed allowing blocks only at the _end_ of expressions, and not in the middle. OTOH, I may just be projecting the interpretation of that statement that suits me best. :) To answer some of the other points that have been raised: Steven:

Did I understand correctly that you prohibited using `if`, `while` and other block statements inside the do block? No, you misunderstood. What you are thinking of is the fact that the test expression in an if block, or a while block, cannot have an appended block. A block is really just syntactic sugar for a def - it lacks only a name, and the ability to be decorated. It can have a docstring, it can be a generator, etc.

Steven:

Personally, the only time I've ever missed the ability to create multi-line lambdas was when I had a series of code blocks like this:

try: BLOCK except exception, e: process_error(e)

where the BLOCK was different in each one, and I really wanted to factor out the common code into a function "process" and pass each BLOCK as an argument:

while condition: process(BLOCK1) process(BLOCK2) if something: process(BLOCK3)

sort of thing.

Do you mean something like this? def safely(body): try: body() except exception, e: process_error(e) while condition: safely() do: BLOCK1 safely() do: BLOCK2 if something: safely() do: BLOCK3 I don't claim that a pattern like that will work in all cases, because the body of the blocks have their own local scope. If you need to set a variable in the enclosing scope, it might not do what you want (but we do have "nonlocal" now, so that helps). Carl:

That said, I have to insist that what you've proposed as

f() do: BLOCK

be written instead as

f(&) do: BLOCK

Just magically inserting the block as the last argument to a call is something I hate about Ruby

Sure, that's OK with me. I also expected the argument that the empty parentheses after "do" should not be optional, for symmetry with def, and that's fine with me to. I was just trying to keep the syntax as "lightweight" as possible for the simplest cases. Stephen:

Some people feel that giving names to a cognitive chunk is an essential part of the process of chunking. They are made uncomfortable, and often confused, by code which is named only by its own source (aka "blocks"). Others don't feel that way. The former oppose this kind of proposal strongly, the latter support it more or less actively.

Agreed, and I do have a slight nagging worry that blocks could be abused, in that people could use them in cases where a named function would be better (and I agree that there will always be many such cases). But then I remember that "we're all adults here", and that in practice people can just as easily, and often do, give useless names to small functions, and then I don't worry about it so much anymore. Carl:

If it's the lack of function names and docstrings

Python 2.7a0 (trunk, Jul 12 2009, 07:54:12) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.

...

...

def f(b): return b.__doc__ ... f() do: ... "Look, ma, I'm a docstring!" ... "Look, ma, I'm a docstring!"

Overall, I think we're at the point where most of us agree that this is a very subjective matter, and it comes down to taste. Guido has a well-recorded history of saying (paraphrasing) "just write a def with a name and quit whining!", so I suppose the odds for this proposal are not great. But I have certainly enjoyed the discussion, so thanks to everyone who has participated. Chris Perkins

On Wed, Jul 15, 2009 at 8:30 AM, Aahz wrote:

On Wed, Jul 15, 2009, Paul Moore wrote:

...

Maybe there's value in having a "language design principles" PEP where pronouncements like this could be recorded, for the benefit of people proposing new features?

Sure, although PEP 20 seems like an obvious spot.

Also, PEP 3099, although it's more about prohibitions than principles: http://www.python.org/dev/peps/pep-3099/ Cheers, Chris -- http://blog.rebertia.com

Steven D'Aprano wrote:

The key=& seems like a mysterious Perlism -- what's the & operator doing in there? (Admittedly, it *might* be nice to be able to write things like reduce(&, alist) instead of reduce(operator.and_, alist). But that's another story.) "do(item):" confuses me, because it looks like the start of a do loop, but it isn't.

The thing about this debate that confuses me is why was the @ decorator ever approved in the first place? It's line-noise, and worse, it's line-noise with a meaning completely different from any other language, ever. On top of that if you do help(), you get nothing [Ed: in Python <2.6. Good job, whoever fixed this!], and Googling is worthless. When I first learned Python, I was very confused by decorators--in part because I didn't know "decorator" was the term to Google for when trying to figure out the meaning of "@"! And yet, @ was approved, in spite of its worthlessness (it literally adds no power to the language, just sugar) and strangeness because of the perceived benefits to readability. And in my opinion, there are similar cases here, where readability is improved by having the definition of a function come after the expression to which the function will be passed. Yet, there's a lot of resistance to this, and I'm not entirely sure why. If it's just a problem with using "&" and "do" other line-noise and keywords can be proposed to substitute for it. $ has no meaning in Python, for example. If it's the lack of function names and docstrings, there's no reason these lambda-like-thingies can't have support for names and docstrings added (as in my old proposal, for example). But I feel like there's some deeper reason people don't think this is a readability win. What is it? -- Carl

On 7/14/09, Carl Johnson wrote:

Steven D'Aprano wrote:

The thing about this debate that confuses me is why was the @ decorator ever approved in the first place?

It wasn't, for quite a while. I believe it was proposed for 2.2. I'm sure it was proposed for 2.3. Applying it to classes as well as functions was in the working patch -- but waited until another version just in case.

And yet, @ was approved, in spite of its worthlessness (it literally adds no power to the language, just sugar) and strangeness because of the perceived benefits to readability.

I have a bit of an unusual perspective; I ended up writing some drafts of the PEP, but not because I wanted decorators. Rather, I wanted to avoid a really bad syntax. (@ wasn't my favorite, but it is way better than several of the earlier proposals -- particularly in terms of what it doesn't assume about the future.) Almost any syntax is a big improvement *for*the*relevant*use*cases*. The concern is what it would do to python in general, and which future alternatives it would cut off.

And in my opinion, there are similar cases here, where readability is improved by having the definition of a function come after the expression to which the function will be passed.

Some of the examples looked like the order was an improvement. But are there huge areas where this is needed all over the place? (Extension frameworks such as Cocoa were mentioned for decorators.) Is it useful across several types of programming? (I see it as obvious for callbacks, but maybe not so clearcut elsewhere.) What is the cost of not doing it? With decorators, that cost was often delaying the wrapper for 20-30 lines, and then having to repeat the function name three times, and then using a code style that does confuse some people. In this case, I think the cost of not doing it is more like "Think up a name for the 2-3 line function, and then move the definition up ahead of the usage." Moving the definition is annoying, but not so horrible with small functions, and there is no need to repeat names. (I suppose you could argue that there is one repetition, if the function will never be used elsewhere.) What would the proposed syntax cost? In the case of @, that cost was reduced to burning a punctuation character, and explaining that functions can be annotated -- and the annotation is more than a comment. For the currently proposed syntax, it confuses several issues with suites (blocks). It sounds like the parser can probably handle it, but it isn't as clear that humans can. It might well preclude later block extensions, including more powerful versions, such as thunks or macros. (And it still burns a punctuation character anyhow.)

Yet, there's a lot of resistance to this, and I'm not entirely sure why. If it's just a problem with using "&" and "do" other line-noise and keywords can be proposed to substitute for it.

Yes. Or "using:", of course. The catch is that those are all so generic that this is only one of a zillion possible meanings -- and for most people, not the most obvious. A more specific keyword might have a better chance. (Not a *good* chance -- there is quite a high bar for keywords -- but still *better*.)

$ has no meaning in Python, for example.

I think it starts to with string templates. (And I think "!" is used by some popular extensions, such as scipy, and I think @ interfered with the leo editor, and ...) I'm not saying it is ambiguous to the parser, but the universe of unused ascii-punctuation isn't really all that huge.

If it's the lack of function names and docstrings, there's no reason these lambda-like-thingies can't have support for names and docstrings added (as in my old proposal, for example).

That wouldn't help unless people actually used the names and docstrings -- which again brings up the question about why existing use of functions as first class objects isn't enough.

But I feel like there's some deeper reason people don't think this is a readability win. What is it?

Cost-benefit. It is harder to see the costs, because they're spread out thinly all over the language, while the benefits are concentrated. But those costs are important. In this case, it looks like the benefits are smaller than for decorators, and the costs (at least with the proposed syntax) are higher. And decorators were not a shoe-in. -jJ

Paul Moore wrote:

(note that I could make my version look a bit nicer by a "from functools import partial", so you're still not allowed to argue aesthetics :-))?

Make that a "from functools import partial as do" and you get: @do(fn) def _(...): stmt1 stmt2 @do(fn, a, b, c) def _(...): stmt1 stmt2 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------

Chris Perkins wrote:

On Tue, Jul 14, 2009 at 7:41 AM, Carl Johnson wrote:

...

This idea seems to be almost identical to my thread from November, "Proposal for Ruby-style anonymous block functions (that don't kill the indention)": http://mail.python.org/pipermail/python-ideas/2008-November/002340.html

Cool! I had not seen that thread, but the fact that you and I have proposed nearly the same thing independently increases the odds that I'm not just nuts. :)

...

The real difference is that your proposal uses & (which is a bad idea, since & already has a meaning: __and__) and in

foo() do: BODY

it implicitly injects the BODY function into the args of foo, which violates Explicit Is Better Than Implicit.

I considered the use of a "magic" placeholder character to be an unfortunate but necessary evil - the alternative seemed to be to restrict the block/def-expression to becoming the last argument to the function call that it follows, and that was just too restrictive. I have no particular attachment to "&" - but note that there is no ambiguity with the bitwise and operator, as far as I can tell. Eg. this works:

...

...

...

def f(a): return a() ... 3 & f(&) do: return 5 1

(not that you would write real code like that...)

[snip] Another possibility is to permit a local name (it's not an anonymous function any more!) so that you can have multiple functions: 3 & f(foo) def foo: return 5 3 & f(foo, bar) def foo, bar: return 5 and: return 6 Anyway, how would you handle condition expressions, as used in 'if' and 'while' statements? /They/ also expect a terminating colon and an indented block.