Proposal for function expressions
I have a proposal for a language feature - a limited form of function- definition expression, similar (superficially) to Ruby's blocks. The big problem with making "def" an expression is the indentation of the function body. You can't just embed an indented block of code into the middle of an expression. The solution I propose (inspired by Ruby) is to allow a block of code to be appended to the end of an expression in certain, limited circumstances. The block is simply syntactic sugar for a local anonymous function definition which is passed as an argument to the function call that it follows. First, a simple example to give the broad strokes: foo() do: BODY is equivalent to: def ANON(): BODY foo(ANON) where the name ANON is a placeholder for illustration - it is never actually bound in the local namespace. Specifically, here is what I propose: * A new syntactic construct, the "block", which consists of: * the keyword "do" (alternative: "def", repurposed) * an optional argument list * a colon * an indented block * A block is only allowed immediately following a call (or indexing expression? see below), on the same line as the closing right brace * The block defines an anonymous local function which is passed as an argument to the call that it follows. * The block is passed as the last argument to the function by default, but this can be changed by putting a placeholder, the character "&", into the parameter list of function. This is best illustrated by example: # 1) In the following, foo recieves TWO arguments: 23 and a callable, in that order. foo(23) do: pass # 2) Here, foo also recieves TWO arguments: a callable and 23, in that order. foo(&, 23) do: pass Why "&"? No particular reason - just because Ruby uses "&" for something similar (but not the same). Why do we need the "&" feature? Consider this example: map(&, my_list) do(item): return do_something_to(item) This also works for keyword arguments: foo(a=3, b=&, c=7) do(): whatever() To make this syntax work, we need several restrictions on where a block is allowed. Intuitively, the rules are: * If the line following "do" would normally be indented, then a block is not allowed. * If the line following the "do" would be one on which leading indentation is insignificant, a block is not allowed. To clarify, the first rule means that this is not allowed: if foo() do: # are we in the body of the block, or of the if? The second rule means that this is not allowed: bar(23, foo() do: body_of_block() ) # closing brace of call to bar Here are some properties of blocks that may not be immediately obvious: * Blocks are a feature of the call site, and do not affect function definitions. In other words, there is no such thing as a "function expecting a block", as there is in Ruby. From the callee's point of view, he has simply been passed a callable as an argument. * Blocks can be used with any existing callable that expects to be passed a function as one of its arguments. OK, let's move on to the fun part: Motivating Examples. ################### # Networking dfr = twisted.whatever(...) dfr.addCallback() do(result): handle(result) dfr.addErrback() do(err): handle_err(err) ################### # GUI (for some hypothetical library) b = my_widget.add_button('Go') b.on('click') do(evt): if evt.ctrl_key: do_something() else: do_other_stuff() ################### # Routing HTTP requests map.match('/') do(req): return render_home(req) map.match('/about') do(req): return render_about(req) map.match(/(?P<controller>\w+)\/$/) do(req, **kw): return handlers[kw['controller']](req) ################### # Threads thread.start_new() do: do_some_work() ################### # Reduce the temptation to cram too much into a lambda. # Sort a list of filenames that look like this: # Xyz-1.1.3.tgz, acbd-4.7.tgz, Xyz-1.2.5.tgz, ... my_list.sort(key=&, reverse=True) do(item): name, _ = os.path.splitext(item) root, _, version = name.partition('-') parts = version.split('.') return (root.lower(),) + tuple(map(int, parts)) stuff = filter(&, my_list) do(item): # some code here... return pred(item) stuff = re.sub(rx, &, s) do(m): t = m.group(0) if cond(t): return t.upper() else if othercond(t): return t.lower() else: return '' ################### # Faking switch switch(expr) do(case): case(int) do: stuff() case(str, bytes) do: other_stuff() switch(value) do(case): case[0:10] do(val): handle_small(val) case[10:100] do(val): handle_medium(val) case.default() do(val): handle_big(val) ################### # Overloaded functions (adapted from PEP 3124) @overload def flatten(when): when(basestring) do(ob): yield ob when(Iterable) do(ob): for o in ob: for ob in flatten(o): yield ob when.otherwise() do(ob): yield ob # later: flatten.when(file) do(f): for line in f: yield line ################### # Sort-of a "with(lock)" block, but running the body in a thread. pool.run_with(my_lock) do: # lock is held here work() other_work() # in parallel ################### Well, that should give you the general idea. Some final random points: * As proposed, blocks are a new kind of beast in Python - an indented block that is part of an expression, rather than being a statement. I think that by constraining a block to be the last thing in an expression, this works out OK, but I may be missing something * Since writing this, I have started to lean towards "def" as the keyword, rather than "do". I used "do" in the examples mostly to make them look like Ruby, but I suspect that making Python "look like Ruby" is not a design goal for most people :) * Terminology: "block"? "def expression"? "anonymous def"? "lambda++"? * Idea: Could we relax the rules for where a block is allowed, removing the constraint that it follows a call or indexing expression? Then we could do, for example, this: def f(): return def(args): pass * Would a proliferation of nameless functions be a nightmare for debugging? Implementation is here: http://bitbucket.org/grammati/python-trunk/ Patch is here: http://bugs.python.org/issue6469 To be honest, I'm not entirely sure if the patch is in the right format - it comes from hg, not svn. Chris Perkins
Chris Perkins wrote:
I have a proposal for a language feature - a limited form of function- definition expression, similar (superficially) to Ruby's blocks.
The big problem with making "def" an expression is the indentation of the function body. You can't just embed an indented block of code into the middle of an expression. The solution I propose (inspired by Ruby) is to allow a block of code to be appended to the end of an expression in certain, limited circumstances. The block is simply syntactic sugar for a local anonymous function definition which is passed as an argument to the function call that it follows.
First, a simple example to give the broad strokes: foo() do: BODY
is equivalent to:
def ANON(): BODY foo(ANON)
Add del anon and the equivalence is almost exact. Call all one-off functions '_' and there is no need to ever delete the name and you have def _(): body foo(_) which is nearly identical to your proposed foo(&). In any case, ideas similar to this have been proposed and discussed ad nausem and rejected. Guide decided for Python 3 to neiher delete function expressions, which he seriously considered, nor to expand them. Feel free to peruse the archives and PEPs. Terry Jan Reedy
Terry Reedy wrote:
Chris Perkins wrote:
First, a simple example to give the broad strokes: foo() do: BODY
is equivalent to:
def ANON(): BODY foo(ANON)
Add del anon and the equivalence is almost exact. Call all one-off functions '_' and there is no need to ever delete the name and you have
def _(): body foo(_)
which is nearly identical to your proposed foo(&).
You can even use decorator notation to move the call before the function body: @foo def thunk(): BODY (Note that most code block ideas haven't actually made it to the PEP stage - the only one along those lines that I can see in PEP 0 is PEP 359's "make" statement, which had more to do with metaclasses than code blocks. That said, I recall assorted code block based ideas being thrown around in the PEP 340/342/343/346 discussions that eventually lead to the introduction of PEP 343's with statement and PEP 342's generator enhancements) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
On Sun, Jul 12, 2009 at 4:11 PM, Chris Perkins<chrisperkins99@gmail.com> wrote:
First, a simple example to give the broad strokes: foo() do: BODY
is equivalent to:
def ANON(): BODY foo(ANON)
So the named and anoymous functions don't share scope in any way? Then what is the advantage? Is putting the call ahead of the def that valuable for making the code clear?
Implementation is here: http://bitbucket.org/grammati/python-trunk/ Patch is here: http://bugs.python.org/issue6469 To be honest, I'm not entirely sure if the patch is in the right format - it comes from hg, not svn.
If using hg for patches is a problem, then the conversion from svn to hg is in for some rough sledding. -jJ
On Sun, Jul 12, 2009 at 8:31 PM, Jim Jewett<jimjjewett@gmail.com> wrote:
So the named and anoymous functions don't share scope in any way?
Then what is the advantage? Is putting the call ahead of the def that valuable for making the code clear?
Yes, exactly - or at least, I think it is. I have found that putting the call before the def adds a surprising amount readability. I came to this conclusion from doing primarily JavaScript for the last couple of years - there, you have a choice of whether to predefine a local function, or to put one inline. eg: var callback = function(result) { // 10 lines of code... }; make_ajax_call(url, callback); vs. make_ajax_call(url, function(){ // 10 lines of code... }); I have come to hate the former style, and find myself frequently refactoring it into the latter, because the main thrust of what this code does is summed up by "make_ajax_call(url, ..)", so that is what I want to see first - not way down at the end, like an afterthought. In other words, I think code that puts things in the opposite order from the way you think of them is harder to read. Another example: if I'm thinking that what I want to do is "substitute some stuff in a string", then I want to start by typing/reading "re.sub(...)", and not "def some_made_up_name(...):" re.sub(pat, &, s) do(m): # several lines of code vs. def temp(m): # several lines of code re.sub(pat, temp, s) The latter relegates the key line of code to the very last line, making it harder to see at a glance what it does. So yes, the point of this really is just to allow you to write code "in the right order". I guess I should have made all this clearer in my original email :) Chris Perkins
On Mon, Jul 13, 2009 at 06:51:27AM -0400, Chris Perkins wrote:
var callback = function(result) { // 10 lines of code... }; make_ajax_call(url, callback);
vs.
make_ajax_call(url, function(){ // 10 lines of code... });
For me, the latter is unreadable (it's hard to see where is the inner block, and where is the outer; where are the make_ajax_call's arguments and where are the function's arguments and body) even if the function is 1-2 line(s) long; if the function's body is longer the style becomes completely unacceptable for my not so sharp eyes and brain. Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd@phd.pp.ru Programmers don't die, they just GOSUB without RETURN.
Dear Pythonistas, It's my first post in the list, so let me introduce myself. My name is Jan Kaliszewski, I am a composer (studied in Frederic Chopin Academy of Music in Warsaw) and a programmer (currently working in Record System company on Anakonda -- ERP system being programmed in Python). Comming to the matter... 13-07-2009 Chris Perkins <chrisperkins99@gmail.com>:
In other words, I think code that puts things in the opposite order from the way you think of them is harder to read.
Another example: if I'm thinking that what I want to do is "substitute some stuff in a string", then I want to start by typing/reading "re.sub(...)", and not "def some_made_up_name(...):"
re.sub(pat, &, s) do(m): # several lines of code
vs.
def temp(m): # several lines of code re.sub(pat, temp, s)
The latter relegates the key line of code to the very last line, making it harder to see at a glance what it does.
So yes, the point of this really is just to allow you to write code "in the right order".
I like the idea of '&' (or another character/word in that role) -- but maybe it'd be enough to extend an existing feature: decorators with it, and to add possibility to use that special character/word as a function name after 'def'? E.g.: ################### # Networking dfr = twisted.whatever(...) # @dfr.addCallback(&) could be equivalent @dfr.addCallback def &(result): handle(result) @dfr.addErrback def &(err): handle_err(err) ################### # GUI (for some hypothetical library) @my_widget.add_button('Go').on('click', &) def &(evt): if evt.ctrl_key: do_something() else: do_other_stuff() # or, if we want to keep the result: @my_widget.add_button('Go').on('click', &) def button_clicked(evt): if evt.ctrl_key: do_something() else: do_other_stuff() ################### # Threads @thread.start_new: def &(): do_some_work() Best regards, Jan Kaliszewski
This seems to be a nice idea. +0.5 -panzi On 07/13/2009 01:40 PM, Jan Kaliszewski wrote:
Dear Pythonistas,
It's my first post in the list, so let me introduce myself. My name is Jan Kaliszewski, I am a composer (studied in Frederic Chopin Academy of Music in Warsaw) and a programmer (currently working in Record System company on Anakonda -- ERP system being programmed in Python).
Comming to the matter...
13-07-2009 Chris Perkins <chrisperkins99@gmail.com>:
In other words, I think code that puts things in the opposite order from the way you think of them is harder to read.
Another example: if I'm thinking that what I want to do is "substitute some stuff in a string", then I want to start by typing/reading "re.sub(...)", and not "def some_made_up_name(...):"
re.sub(pat, &, s) do(m): # several lines of code
vs.
def temp(m): # several lines of code re.sub(pat, temp, s)
The latter relegates the key line of code to the very last line, making it harder to see at a glance what it does.
So yes, the point of this really is just to allow you to write code "in the right order".
I like the idea of '&' (or another character/word in that role) -- but maybe it'd be enough to extend an existing feature: decorators with it, and to add possibility to use that special character/word as a function name after 'def'?
E.g.:
################### # Networking
dfr = twisted.whatever(...)
# @dfr.addCallback(&) could be equivalent @dfr.addCallback def &(result): handle(result)
@dfr.addErrback def &(err): handle_err(err)
################### # GUI (for some hypothetical library)
@my_widget.add_button('Go').on('click', &) def &(evt): if evt.ctrl_key: do_something() else: do_other_stuff()
# or, if we want to keep the result:
@my_widget.add_button('Go').on('click', &) def button_clicked(evt): if evt.ctrl_key: do_something() else: do_other_stuff()
################### # Threads
@thread.start_new: def &(): do_some_work()
Best regards, Jan Kaliszewski
On Mon, Jul 13, 2009 at 6:51 AM, Chris Perkins<chrisperkins99@gmail.com> wrote:
var callback = function(result) { // 10 lines of code... }; make_ajax_call(url, callback);
vs.
make_ajax_call(url, function(){ // 10 lines of code... });
This is common in Javascript, but still makes me cry. I would rather see something like this: function main() { make_ajax_call(url, ajax_call); } function ajax_call() { // 10 lines of code } This way ajax_call can be in a different file and can be used in other code without duplication.
I have come to hate the former style, and find myself frequently refactoring it into the latter, because the main thrust of what this code does is summed up by "make_ajax_call(url, ..)", so that is what I want to see first - not way down at the end, like an afterthought.
I think you are refactoring in reverse. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek
This idea seems to be almost identical to my thread from November, "Proposal for Ruby-style anonymous block functions (that don't kill the indention)": http://mail.python.org/pipermail/python-ideas/2008-November/002340.html The real difference is that your proposal uses & (which is a bad idea, since & already has a meaning: __and__) and in foo() do: BODY it implicitly injects the BODY function into the args of foo, which violates Explicit Is Better Than Implicit. I agree though that sometimes rather than doing def spam(): BODY result = foo(spam) you would rather do result = foo(spam*) *To Be Defined on the next line, OK! def spam(): BODY because it reads better. As I understand it, this was the reason the decorator syntax was invented. Basically, @classmethod def foo(): BODY reads better than def foo(): BODY foo = classmethod(foo) so decorators are just syntatic sugar to improve readability. So, I agree with the basic motivation of your proposal, but I think the specifics of it aren't as good as my old one (although, I *would* think that, wouldn't I?) and from experience, I expect the BDFL to come and give you -2 in a couple days if the thread keeps up… In terms of refining my old proposal, it strikes me that ! isn't used in Python currently. If we have to use a sigil of some sort, we could try: new_list = sorted(old_list, key=!keyfunc) def keyfunc(item): normalize data… or something to that effect with the general semantic being that ! means "here comes a variable name, I will define this variable on the next line, please wait for it!" 2-cents-ly-yrs, -- Carl Johnson
On Tue, Jul 14, 2009 at 7:41 AM, Carl Johnson<cmjohnson.mailinglist@gmail.com> wrote:
This idea seems to be almost identical to my thread from November, "Proposal for Ruby-style anonymous block functions (that don't kill the indention)": http://mail.python.org/pipermail/python-ideas/2008-November/002340.html
Cool! I had not seen that thread, but the fact that you and I have proposed nearly the same thing independently increases the odds that I'm not just nuts. :)
The real difference is that your proposal uses & (which is a bad idea, since & already has a meaning: __and__) and in
foo() do: BODY
it implicitly injects the BODY function into the args of foo, which violates Explicit Is Better Than Implicit.
I considered the use of a "magic" placeholder character to be an unfortunate but necessary evil - the alternative seemed to be to restrict the block/def-expression to becoming the last argument to the function call that it follows, and that was just too restrictive. I have no particular attachment to "&" - but note that there is no ambiguity with the bitwise and operator, as far as I can tell. Eg. this works:
def f(a): return a() ... 3 & f(&) do: return 5 1
(not that you would write real code like that...)
I agree though that sometimes rather than doing
def spam(): BODY result = foo(spam)
you would rather do
result = foo(spam*) *To Be Defined on the next line, OK! def spam(): BODY
because it reads better. As I understand it, this was the reason the decorator syntax was invented. Basically,
@classmethod def foo(): BODY
reads better than
def foo(): BODY foo = classmethod(foo)
so decorators are just syntatic sugar to improve readability.
So, I agree with the basic motivation of your proposal, but I think the specifics of it aren't as good as my old one (although, I *would* think that, wouldn't I?) and from experience, I expect the BDFL to come and give you -2 in a couple days if the thread keeps up…
In terms of refining my old proposal, it strikes me that ! isn't used in Python currently. If we have to use a sigil of some sort, we could try:
new_list = sorted(old_list, key=!keyfunc) def keyfunc(item): normalize data…
or something to that effect with the general semantic being that ! means "here comes a variable name, I will define this variable on the next line, please wait for it!"
I prefer not repeating the function name, I prefer putting the "do:" (or "def:") on the same line (makes it clearer that the indented block is effectively part of the expression that it follows), and I like the fact that the function has no name - it tells you immediately that "this function is only used within this one call - don't go looking for uses of it later on, because they won't be there". Other than that, it seems we are in perfect agreement on the feature. It also seems that your earlier proposal met with the same rabid ambivalence that mine has so far :) A couple of interesting quotes from the discussion of your proposal: Carl: "this proposal is not about adding power to Python, just making things more readable" Exactly! Greg: "You could just allow any expression-statement to be followed by a block, and sort out whether it makes sense later on." That's exactly how I ended up implementing it. Carl, if you still have any interest in this feature, you should try out the patch - I think you'll be the first. :) Also I'll be happy to give you commit privileges to the hg repo where I did the implementation, in case you want to try to come up with some middle ground between my proposal and yours: http://bitbucket.org/grammati/python-trunk/ Failing that, it looks like our proposals will die the same not-so-slow death for lack of interest. Oh well. Chris Perkins
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
Failing that, it looks like our proposals will die the same not-so-slow death for lack of interest. Oh well.
OK, so I'm not accused of "lack of interest"... :-) First of all, I have never felt the need for a construct like this. So it adds nothing but complexity to the language for me. But assuming that in the future, I do find a need for this, or there's a large group of other users who would find this useful, please can you address the following: Your fn() do (...): stmt1 stmt2 is equivalent to @fn def _(...): stmt1 stmt2 Why is your proposal better (ignoring "I like it better", as subjective preferences other than Guido's don't get the language changed)? For the case of function calls with arguments, why is fn(a, b, c) do(...): stmt1 stmt2 better than @functools.partial(fn, a, b, c) def _(...): stmt1 stmt2 (note that I could make my version look a bit nicer by a "from functools import partial", so you're still not allowed to argue aesthetics :-))? That leaves your general cases where you stick an "&" somewhere in the call as a placeholder. Only a couple of your motivating examples used that form, and to be honest I didn't find them very convincing. By the time you're getting that complex, I really do think you're trying to cram too much into one statement (and I know that's a subjective statement, sorry :-)) The same applies to the "x = fn(...) do (...)" cases that you include in the example, but don't explicitly state in your specification. I don't see any objective reason to think your form is clearer - yes, "put the assignment, which is the main thing, up front" is an argument in your favour, but "don't put too many things together just to avoid thinking of a name" is an equally valid against. OK, I hope that helps. I'm still against the idea, but I hope this gives you some idea why - it's not simply indifference. A construct like this needs to bring something better than mere improved readability to the language, in my view. Paul.
On Tue, Jul 14, 2009 at 11:04 AM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
Failing that, it looks like our proposals will die the same not-so-slow death for lack of interest. Oh well.
OK, so I'm not accused of "lack of interest"... :-)
First of all, I have never felt the need for a construct like this. So it adds nothing but complexity to the language for me. But assuming that in the future, I do find a need for this, or there's a large group of other users who would find this useful, please can you address the following:
Your
fn() do (...): stmt1 stmt2
is equivalent to
@fn def _(...): stmt1 stmt2
Why is your proposal better (ignoring "I like it better", as subjective preferences other than Guido's don't get the language changed)?
I agree that decorators can handle some of the use cases - I anticipated that argument, and it is indeed a valid one. Nevertheless, I think that's a bit a perversion of the the purpose of decorators. And as you point out below, decorators are significantly more limited in what they can do. And besides that, "I like it better" :)
For the case of function calls with arguments, why is
fn(a, b, c) do(...): stmt1 stmt2
better than
@functools.partial(fn, a, b, c) def _(...): stmt1 stmt2
(note that I could make my version look a bit nicer by a "from functools import partial", so you're still not allowed to argue aesthetics :-))?
Yes, again, that does work, but it's getting uglier (yes, I know, subjectiveness again), and it forces the callable to be the last argument, which is limiting.
That leaves your general cases where you stick an "&" somewhere in the call as a placeholder. Only a couple of your motivating examples used that form, and to be honest I didn't find them very convincing. By the time you're getting that complex, I really do think you're trying to cram too much into one statement (and I know that's a subjective statement, sorry :-))
Indeed, like any language construct it could be used for evil. I prefer to think of it as enabling the programmer to write clearer code in some cases - trying to prevent him from writing unclear code is, in my opinion, a lost cause. Some programmers will find a way to write unclear code no matter what tools you give them.
The same applies to the "x = fn(...) do (...)" cases that you include in the example, but don't explicitly state in your specification. I don't see any objective reason to think your form is clearer - yes, "put the assignment, which is the main thing, up front" is an argument in your favour, but "don't put too many things together just to avoid thinking of a name" is an equally valid against.
For me, avoiding thinking of a name was never the primary motivation - putting things in the "right order" is. But I do believe that there are situations where the context that a block of code sits in tells far more about its purpose that a name ever could - in those circumstances, a forced function name can be useless at best, and distracting clutter at worst. For example: y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper() I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does. I also think that the current state of affairs encourages poorer readability - I mean, admit it, today you would probably write that snippet like this: y = sorted(x, key=lambda item: item.split('-')[0].upper()) which to me loses clarity both by being crammed into one line, and by not being able to create the local variable "name", which tells you quite a bit about why the the item is being chopped up the way it is.
OK, I hope that helps. I'm still against the idea, but I hope this gives you some idea why - it's not simply indifference.
It does help - thank you for your comments, Paul.
A construct like this needs to bring something better than mere improved readability to the language, in my view.
Aha! You said "improved readability"! So I'm going to conveniently forget everything else you said and write your name in the "pro" column. :) Seriously, I guess this is where we differ - I think that improved readability _is_ a sufficient goal unto itself. The only question is whether other people (OK, one other person in particular) thinks it improves readability, and if so, then by how much. Chris Perkins
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
For me, avoiding thinking of a name was never the primary motivation - putting things in the "right order" is. But I do believe that there are situations where the context that a block of code sits in tells far more about its purpose that a name ever could - in those circumstances, a forced function name can be useless at best, and distracting clutter at worst. For example:
y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper()
I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.
That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me). By contrast. def uc_name(item): name = item.split('-')[1] return name.upper() y = sorted(x, key=uc_name) is a lot clearer to me. And it would be even better if I knew what was going on with the splitting of items on dashes, and taking the second bit, was all about. Then, I could give the function a more meaningful name. [[An admission here - when I first wrote that example, I spelled the function name differently in the 2 places. So my "see how much better my way is" example was wrong - in a way yours couldn't be! In my defence, the error would have come up as soon as I ran the code, and I could say that if I had thought of a better name it wouldn't have happened - but nevertheless. In the spirit of friendly debate, I'll give you that argument for your case free of charge :-)]] My point is that *technically* I can see what the 2-line block of code does - but without giving it a name, I don't have a clue what it's *for*. But I don't imagine we're ever going to agree on this. Let it stand that (without implying any criticism - people differ) your idea of readable code is confusing and difficult to follow for me. Hence my distrust of readability arguments :-)
A construct like this needs to bring something better than mere improved readability to the language, in my view.
Aha! You said "improved readability"! So I'm going to conveniently forget everything else you said and write your name in the "pro" column. :)
Good try, but you don't win that easily :-) I don't think this construct improves readability. Personally, I think it *harms* readability (although I concede that like anything, it could be used tastefully). What I was saying was that your *only* argument was readability, and I don't think that is enough of an argument in itself.
Seriously, I guess this is where we differ - I think that improved readability _is_ a sufficient goal unto itself. The only question is whether other people (OK, one other person in particular) thinks it improves readability, and if so, then by how much.
It has to improve readability, uncontroversially, in enough cases to justify the cost in terms extra of language complexity (both for implementation, and definition/teaching). To demonstrate the benefits, a good place to look is often the standard library. If your proposed change can be demonstrated to improve the code shipped with Python in the stdlib, in a number of places, people will start to listen. Also, apart from the basic cost of any change (the fundamental barrier to entry, if you like) the cost of your change is higher because there are obscure corner cases that aren't easy to explain - witness questions here about interaction of this form with if statements, etc. This is a unique construct in that it's the only indentation-sensitive multi-line element of an expression. People don't have an intuition about how it "should" work, because there's nothing similar to extrapolate from. Ultimately, that's why this idea (in one form or another) has been around for so long and never been accepted - nobody has ever managed to come up with an idea that prompted the "oh, yes, obviously!" reaction in people needed to confirm that it's the "right" solution. I'd like to think that one day, someone will come up with the ideal answer which will finally address all these "extended lambda"/"code block" needs. But I don't think it's happened yet. Sorry :-) Paul.
It may be possible to work out the grammar so that & is never ambiguous between "insert block here" and "and these two things together." I'd need to think about it some more to be sure. One advantage of using & as a sigil is that it jives with what Rubyists are used to. That said, I have to insist that what you've proposed as f() do: BLOCK be written instead as f(&) do: BLOCK Just magically inserting the block as the last argument to a call is something I hate about Ruby, and I think their experience has shown that it's a mistake, since it hurts flexibility. EIBTI, etc. To those claiming this can all be done with decorators, see another one of my old threads: "Allow Lambda Decorators" http://mail.python.org/pipermail/python-ideas/2009-February/002787.html It is true that instead of using new_list = sorted(old_list, key=&) do item: name = item.split("-")[0] return name.lower() one might do @list_sorted(oldlist) def new_list(item): name = item.split("-")[0] return name.lower() with list_sorted defined as def list_sorted(seq): def inner(f): return sorted(seq, key=f) return inner But the problem with this is that the reader of your code will see the "@" and think "here comes the definition of a callable." It's only after a lot of thinking that the reader will realize that new_list is a list, not a callable. So, in terms of readability, using decorators, though possible, is probably a mistake. -- Carl Johnson
On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper()
I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.
That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me).
Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation: y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() ) There must be somewhere posted the reason this was rejected but my google-fu is failing me. George [1] http://boo.codehaus.org/Closures
On Tue, Jul 14, 2009 at 8:50 PM, George Sakkis<george.sakkis@gmail.com> wrote:
On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper()
I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.
That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me).
Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:
y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() )
There must be somewhere posted the reason this was rejected but my google-fu is failing me.
IIRC, it screws with the essential statement-expression dichotomy of Python: http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html Cheers, Chris -- http://blog.rebertia.com
On Tue, Jul 14, 2009 at 11:55 PM, Chris Rebert<pyideas@rebertia.com> wrote:
On Tue, Jul 14, 2009 at 8:50 PM, George Sakkis<george.sakkis@gmail.com> wrote:
On Tue, Jul 14, 2009 at 3:16 PM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/14 Chris Perkins <chrisperkins99@gmail.com>:
y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper()
I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.
That's a very telling statement. To me, that code is utterly baffling. (Seriously! Without analysing it line by line, it means nothing to me).
Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:
y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() )
There must be somewhere posted the reason this was rejected but my google-fu is failing me.
IIRC, it screws with the essential statement-expression dichotomy of Python: http://unlimitednovelty.com/2009/03/indentation-sensitivity-post-mortem.html
Thanks, this was it. The important part is Guido's quote "any solution that embeds an indentation-based block in the middle of an expression is unacceptable", so it's basically a design judgement, not an implementation constraint. George
On Tue, 14 Jul 2009 23:50:29 -0400 George Sakkis <george.sakkis@gmail.com> wrote:
Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:
y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() )
There must be somewhere posted the reason this was rejected but my google-fu is failing me.
It gets ugly when carried to - well, extremes is the word I want to use, but is wanting two or three functional arguments inlined really "extreme"? Likewise, what happens if you want to inline a functional argument in an already inlined function? The current proposal seems to avoid that - by limiting the number of function parameters to 1, which is a pretty serious shortcoming. Pretty much all such proposals fail on one of these two issues. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
George Sakkis wrote:
Seconded, there's too much going on ('&', what's "do", a function ?, what's "item" ?, etc.). FWIW the only readable suggestion for multiline lambdas with almost zero learning curve I have seen is the one implemented in Boo, anonymous def and regular indentation:
y = sorted(x, key=def (item): name = item.split('-')[1] return name.upper() )
FWIW, I find that completely ugly and non-Python-esque (unpythonic being perhaps too strong of criticism). :-D Clearly this is an issue which has a lot of good arguments on both sides, and since the final metric has to be "readability of code" it's going to hard to make a knock-down argument for one side or the other. I'm fine with the status quo for now: in Guido's gut we trust. My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem. Just solving that one problem by itself seems unlikely to make it into the language, since it is pure "sugar" and Pythoneers have a well justified wariness toward anonymous functions (why not just name them and be done with it?). Perhaps someone should take a look at the original "with" proposal, or some of the various "do" and "block" proposals. One idea I think about sometimes is having a more convenient way to do something along the lines of metaclass.__prepare__ but for functions… -- Carl Johnson
Carl Johnson wrote:
My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem.
One such extension might be a "where" block. Applied to the current problem: foo(f) where: def f(x): ... Benefits include: * It's pretty much self-explanatory, as it builds on the existing syntax for defining functions, and "where" is used in a similar way in mathematics and some existing programming languages. * You get to give the function a meaningful name if you want. * It's not restricted to a single function argument: def foo(f, g) where: def f(x): ... def g(y): ... * It needn't be restricted to functions: foo(x, y) where: x = a * 42 y = b / 17 Drawbacks include: * Execution happens out of order, although that's an inherent feature of the original proposal as well. There are precedents in the language already, such as list comprehensions and conditional expressions. * Allowing arbitrary statements in the block could possibly be considered detrimental to code readability. To mitigate that, the block contents could perhaps be restricted to binding constructs only. -- Greg
Greg Ewing wrote: Nitpicking:
* It's not restricted to a single function argument:
def foo(f, g) where: def f(x): ... def g(y): ...
Surely, you meant foo(f, g) where: def f(x): ... def g(y): ... Ending a statement that ends in a ":" normally with a "where" is too ambiguous and ought to be prohibited. I need to think about this proposal some more, but it would help solve the issue brought up on the list recently that [f(x) for f in fs if f(x)] is inefficient and [i for i in (f(x) for f in fs) if i] is ugly. new_list = [fx for f in fs if fx] where: fx = f(x) The "where" on the end of the clause does catch your eye and make you think "OK, be aware, something is undefined on this line but will be defined on the next." OTOH, in new_list = [i for i in xs for j in ys if fx] where: fx = f(x) or new_list = [i for i in [j for j in ys if fx?] if fx?] where: fx = f(x) Does the "where" get re-used with each loop of the inner list comprehension or just the outer one? What about total = sum(count for item in l) where: count = item.size if item.size > 0 else 1 It seems like for that case, we definitely want the "where" to apply every time the generator expression loops though, but… Is that feasible? Hmm. -- Carl Johnson
2009/7/15 Carl Johnson <cmjohnson.mailinglist@gmail.com>:
It seems like for that case, we definitely want the "where" to apply every time the generator expression loops though, but… Is that feasible?
To make the form follow the intent, the where block would need to be inside the [...] - which clearly violates the "no indented block structure within expressions" design rule. My initial instinct was that the original form violated that rule, too - but if you interpret the where block as a statement postfix, you're (almost) OK. The (almost) bit is because the where block has to be limited to simple (single-line) statements. But with those restrictions, the semantics become stmt where: a b c => a b c stmt with the proviso that variables defined in a,b,c are only available within a, b, c and stmt. But of course that immediately prompts the question: x = 1 y = x where: x = 2 print x What does that print? x = [] y = 1 where: x.append(1) print x What about that? x = [] y = x z = 1 where: x.append(1) print x, y, x is y And what about that??? And before someone points out Greg's proposal to limit the construct to "binding statements", please define (clearly) what is a "binding statement". And then explain how that helps if I replace x.append(1) with _ = x.append(1) in the above... I sort of like the idea, but it needs much better defined syntax and semantics if it's going to work. Paul.
Hello, 15-07-2009, 09:55 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
One such extension might be a "where" block. Applied to the current problem:
foo(f) where: def f(x): ...
At first glance, +1 from me. 15-07-2009, 12:25 Paul Moore <p.f.moore@gmail.com> wrote:
inside the [...] - which clearly violates the "no indented block structure within expressions" design rule.
My initial instinct was that the original form violated that rule, too - but if you interpret the where block as a statement postfix, you're (almost) OK. The (almost) bit is because the where block has to be limited to simple (single-line) statements.
stmt where: a b c =>
a b c stmt
with the proviso that variables defined in a,b,c are only available within a, b, c and stmt.
I don't like the direction of limiting content of the block to set of singular simple statements. I suppose where-block should simply create a separate nested scope. So...
x = 1 y = x where: x = 2 print x
What does that print?
I tkink it should print 1 -- because, if where-block creates a separate nested scope, its local variables shouldn't leak. But there is another question: what is the value of y: 1 or 2? Or, more genarally: should be the line ended with 'where' included *in* that nested scope or not? I suppose it should, so the final value of y should be 2. Another example: a = 7 b = 8 x = 'X' z = 'zzzzz' foo(a, b, c("It's..."), x) where: # a = input() in Python 3.x :) a = raw_input(z) # -> zzzzz try: a = int(a) except ValueError: a = 0 def b(bar): 'very complex function' print(a, bar) else: def b(bar): 'also very complex function' print(bar, a, bar) def c(*args, **kwargs): 'also also very complex function' print(a, b) # -> 7 8 Best regards, -- Jan Kaliszewski <zuo@chopin.edu.pl>
15-07-2009 o 19:21 Jan Kaliszewski <zuo@chopin.edu.pl> wrote:
I don't like the direction of limiting content of the block to set of singular simple statements. I suppose where-block should simply create a separate nested scope.
...Because IMHO an indented block suggests that the content is a sequence of statements creating a piece of code without unnatural limitations (like in class or function body). *j -- Jan Kaliszewski <zuo@chopin.edu.pl>
On Wed, Jul 15, 2009 at 4:59 AM, Carl Johnson < cmjohnson.mailinglist@gmail.com> wrote:
total = sum(count for item in l) where: count = item.size if item.size > 0 else 1
It seems like for that case, we definitely want the "where" to apply every time the generator expression loops though, but… Is that feasible?
Unlikely, but this is: total = sum(f(item) for item in lst) where: def f(item): return item.size if item.size > 0 else 1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
Carl Johnson wrote:
Surely, you meant
foo(f, g) where: def f(x): ... def g(y): ...
Yes, that's what I meant.
new_list = [fx for f in fs if fx] where: fx = f(x)
No, there would have to be a separate, analogous extension to list comprehensions: new_list = [fx for f in fs if fx where fx = f(x)] or maybe new_list = [fx where fx = f(x) for f in fs if fx] -- Greg
On Wed, Jul 15, 2009 at 2:55 AM, Greg Ewing <greg.ewing@canterbury.ac.nz>wrote:
One such extension might be a "where" block. Applied to the current problem:
foo(f) where: def f(x): ...
I like this proposal much more than any of the previous proposals. It has some of the flavor of Lisp's "let", but puts the important line first instead of at the end. In a nutshell, it says "define these symbols for this one line only". On the downside, for the common case of wanting to define a single function, the body of the function must be indented twice. I suggest the following grammar and meaning, which would be referred to by the assignment, yield, return, and expression statements: where_expression ::= expression_list "where" ":" suite | expression_list It evaluates as follows. [return|yield|x=] expression_list where: suite is roughly equivalent to: def where_expression(): suite return expression_list [return|yield|x=] expression_list() How about the following as additional syntactic sugar for the common one-function case? x = blah(f) where def f(item): body_of_f -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com/>
Why not just use the Haskell approach? foo(x,y) = myfunc(bar) where: myfunc, bar = f(x), g(y) where: f,g = func1, func2 assuming func1 and func2 were previously defined, otherwise another "where" clause could follow. That way, introducing the "where" clause effectively causes foo to be defined as a function. Applying that to the previous example: l = [i for i in myiterator if fi] where: f = f(i) if "myiterator" was previously defined, or even: l = [i for i in myiterator if f(i)] where: myiterator, f = (i for i in xrange(10)), bool if "myiterator" was not previously defined. On Wed, Jul 15, 2009 at 8:49 AM, Daniel Stutzbach<daniel@stutzbachenterprises.com> wrote:
On Wed, Jul 15, 2009 at 2:55 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
One such extension might be a "where" block. Applied to the current problem:
foo(f) where: def f(x): ...
I like this proposal much more than any of the previous proposals. It has some of the flavor of Lisp's "let", but puts the important line first instead of at the end. In a nutshell, it says "define these symbols for this one line only". On the downside, for the common case of wanting to define a single function, the body of the function must be indented twice.
I suggest the following grammar and meaning, which would be referred to by the assignment, yield, return, and expression statements:
where_expression ::= expression_list "where" ":" suite | expression_list
It evaluates as follows.
[return|yield|x=] expression_list where: suite
is roughly equivalent to:
def where_expression(): suite return expression_list [return|yield|x=] expression_list()
How about the following as additional syntactic sugar for the common one-function case?
x = blah(f) where def f(item): body_of_f
-- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC
_______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Gerald Britton
i like defining things prior to running them. One issue, however, is namespace management. i'd like temporary names to be discarded automatically. where: def foo(x,y,z): return x+y+z do: bar = foo(1,2,3) print foo(1,2,3) // foo is undefined! print bar // 6 you could create a local namespace w/ just the 'where' block. where: x = 1 y = x print x // x is undefined! Perhaps we could bind the variables and pass them around. where my_locals: x = 1 y = 2 print my_locals() // {'x': 1, 'y': 2} print x // x is undefined! - Jae On Jul 15, 2009, at 12:55 AM, Greg Ewing wrote:
Carl Johnson wrote:
My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem.
One such extension might be a "where" block. Applied to the current problem:
foo(f) where: def f(x): ...
Benefits include:
* It's pretty much self-explanatory, as it builds on the existing syntax for defining functions, and "where" is used in a similar way in mathematics and some existing programming languages.
* You get to give the function a meaningful name if you want.
* It's not restricted to a single function argument:
def foo(f, g) where: def f(x): ... def g(y): ...
* It needn't be restricted to functions:
foo(x, y) where: x = a * 42 y = b / 17
Drawbacks include:
* Execution happens out of order, although that's an inherent feature of the original proposal as well. There are precedents in the language already, such as list comprehensions and conditional expressions.
* Allowing arbitrary statements in the block could possibly be considered detrimental to code readability. To mitigate that, the block contents could perhaps be restricted to binding constructs only.
-- Greg _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
16-07-2009, 13:49 Nick Coghlan <ncoghlan@gmail.com> wrote, (in thread 'Immemorial desire for "do-while"-likeconstruction'):
Steven D'Aprano wrote:
On Thu, 16 Jul 2009 08:50:06 am Jan Kaliszewski wrote:
Hello,
The issue has been coming back repeatedly:
And not very long ago at that.
So often that there's even a PEP* that documents the pros and cons of different ideas and points out that a viable syntactic challenger to the "while True with inner break" approach has yet to be found.
Maybe the where-statement could be a candidate? Consider an example: while foo(x) and bar() where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY or maybe better: while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY and Daniel's idea of one-def-shortcut also could apply: while setup(x < y + 9 or bzzz): where def setup(cond): SETUP ACTIONS return cond THE LOOP BODY And what about for-loop? for i in smartgen(): where def smartgen(): SOME ACTIONS yield SOMETHING THE LOOP BODY Cheers, -- Jan Kaliszewski <zuo@chopin.edu.pl>
or maybe better:
while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY
Another possible variant: while foo(x) and bar() # no colon where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY But the former has such an advantage that where-block content is indented deeper than loop body -- what reflects that it (where-block) creates nested scope (contrary to loop body). -- Jan Kaliszewski <zuo@chopin.edu.pl>
Jan Kaliszewski wrote:
while foo(x) and bar() where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY
That's kind of clever, although it does require modifying the while-loop syntax, and if you're doing that, there's not a lot of gain over introducing a dedicated loop-and-a-half syntax. Also, once we're allowed 'where' clauses on while statements, people are going to want them on other kinds of statements as well: if a == b where: a = foo() b = blarg() then: ... class Ham(Spam) where: class Spam: ... is: ... def f(x = default) where: default = whatever as: ... try: ... except eels where: eels = something_bad() do: ... Where do we stop?
while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY
I think I liked the first version better. There's something jarring about the unindent half way through without any keyword to mark the boundary. Also the bare 'where' relating to the previous line suggests some weird grammatical gymnastics going on. -- Greg
Greg Ewing wrote:
Jan Kaliszewski wrote:
while foo(x) and bar() where: foo = SOMETHING def bar(): SOMETHING ELSE do: THE LOOP BODY
That's kind of clever, although it does require modifying the while-loop syntax, and if you're doing that, there's not a lot of gain over introducing a dedicated loop-and-a-half syntax.
Also, once we're allowed 'where' clauses on while statements, people are going to want them on other kinds of statements as well:
if a == b where: a = foo() b = blarg() then: ...
class Ham(Spam) where: class Spam: ... is: ...
def f(x = default) where: default = whatever as: ...
try: ... except eels where: eels = something_bad() do: ...
Where do we stop?
while foo(x) and bar(): where: foo = SOMETHING def bar(): SOMETHING ELSE THE LOOP BODY
I think I liked the first version better. There's something jarring about the unindent half way through without any keyword to mark the boundary. Also the bare 'where' relating to the previous line suggests some weird grammatical gymnastics going on.
"do" might be a sufficiently generic word to cover all the cases. while ... where: ... do: ... if a == b where: ... do: ... etc.
Sometimes I use helper functions to make things easier to read. However, I don't like unnecessary function calls since they are expensive in Python compared to other languages. Still I might do this: def foo(x): def bar(y): ... return ... #something depending on y ... return bar(z) If I had a "where" clause I could write: def foo(x): ... return y where: #compute y Using "where" would have the advantage of avoiding a function call, though I suppose it wouldn't save (much) stack space since it needs a new scope anyway. I see the "where" clause as a useful way to abstract a section of code to keep the high level in view, then break it down as you go.
2009/7/17 Gerald Britton <gerald.britton@gmail.com>:
Using "where" would have the advantage of avoiding a function call, though I suppose it wouldn't save (much) stack space since it needs a new scope anyway. I see the "where" clause as a useful way to abstract a section of code to keep the high level in view, then break it down as you go.
You're assuming that the implementation of where is faster than a function call. Given that the defined semantics are in terms of defining then calling a temporary function, this may well be an invalid assumption. (And if it isn't, then can the tricks which get used to speed up the where cause please be used to speed up function calls in general, too - as that would be far more generally useful! :-)) Paul.
On Fri, 17 Jul 2009 11:25:15 pm Gerald Britton wrote:
Sometimes I use helper functions to make things easier to read. However, I don't like unnecessary function calls since they are expensive in Python compared to other languages.
That's only a problem if: (1) Your code actually is too slow; and (2) You have profiled your code and discovered that the biggest contributor to your code's slowness is the cost of calling functions, not the execution time of the functions. Unless both of these cases hold, then I suggest you are guilty of premature optimization. Possibly even pessimation -- given the complexities of CPU caches, memory, swap, etc, it's possible that your code could be *slower*, not faster, due to your "optimization". People's intuitions about what's fast and what's slow in Python code are often completely at odds with reality. Unless you actually have a problem with slow code that needs solving, my advice is to not worry about it. Use good algorithms, use tried and tested fast data structures, and worry about micro-optimizations when you *actually* need them. -- Steven D'Aprano
I often time my code and often find that removing function calls "where" possible actually makes a measurable difference. This is especially the case with little getters and setters in class defs. Anyway, I like the "where" idea not because of real or imagined performance gains, but because of its cleanness when expressing problems. A big use for me would be in list comprehensions. In one project I work on, I see things like: for i in [item in self.someobject.get_generator() if self.important_test(item)] and other really long object references. which I would like to write: mylist = [item in f if g(item)] where: f = self.someobject.get_generator() g = self.important_test To my eyes, the first is harder to read than the second one. Of course I can do this: f = self.someobject.get_generator() g = self.important_test mylist = [item in f if g(item)]: but then "f" and "g" pollute the calling context's namespace. Anyway, I don't have a strong preference, just a "nice to have" feeling. On Fri, Jul 17, 2009 at 10:24 AM, Steven D'Aprano<steve@pearwood.info> wrote:
On Fri, 17 Jul 2009 11:25:15 pm Gerald Britton wrote:
Sometimes I use helper functions to make things easier to read. However, I don't like unnecessary function calls since they are expensive in Python compared to other languages.
That's only a problem if:
(1) Your code actually is too slow; and
(2) You have profiled your code and discovered that the biggest contributor to your code's slowness is the cost of calling functions, not the execution time of the functions.
Unless both of these cases hold, then I suggest you are guilty of premature optimization. Possibly even pessimation -- given the complexities of CPU caches, memory, swap, etc, it's possible that your code could be *slower*, not faster, due to your "optimization".
People's intuitions about what's fast and what's slow in Python code are often completely at odds with reality. Unless you actually have a problem with slow code that needs solving, my advice is to not worry about it. Use good algorithms, use tried and tested fast data structures, and worry about micro-optimizations when you *actually* need them.
-- Steven D'Aprano _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Gerald Britton
To my eyes, the first is harder to read than the second one. Of course I can do this:
f = self.someobject.get_generator() g = self.important_test mylist = [item in f if g(item)]:
but then "f" and "g" pollute the calling context's namespace.
So what? Seriously, what harm does a temporary association do? Especially when one can easily 'clean up' with 'del f,g' if it actually make a functional difference or if one just wants to cater to an obsession. In particular, why is this reason to *PERMANENTLYT pollute the language with an otherwise useless new feature? Unlike namespace associations, deleting language features is really difficult. Terry Jan Reedy
17-07-2009 o 14:12 Ben Finney <ben+python@benfinney.id.au> wrote:
David Stanek <dstanek@dstanek.com> writes:
This thread is a little confusing to me. I am failing to see how the where block makes anything clearer.
+1. I haven't found that any of the examples improve the clarity of the code over using a named function.
And what about that: class SomeClass: ... def summarize_proust(self, objects, args, method_name) "Summarize Proust in 15 seconds" return map(action, objects, args, alts) where: def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result alts = map(weirdtrans, objects) where def weirdtrans(obj): # some transformations not very # useful in more general context ...is more logical and readable for me, than: class SomeClass: ... def _weirdtrans(self, obj): # some transformations useful # only for somefilter method def summarize_proust(self, objects, args, method_name) "Summarize Proust in 15 seconds" def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result alts = map(self._weirdtrans, objects) return map(action, objects, args, alts) 17-07-2009, 22:28 Gerald Britton <gerald.britton@gmail.com> wrote:
which I would like to write:
mylist = [item in f if g(item)] where:
Obviously you ment: "[item for item if g(item)] where:", didn't you? In that particular case you could write: filter(g, f). Anyway still I'd prefer: mylist = filter(g, f) where: f = self.someobject.get_generator() g = self.important_test Than: f = self.someobject.get_generator() g = self.important_test mylist = filter(g, f) where:
then "f" and "g" pollute the calling context's namespace.
+1 Generally one of main applications of 'where' would be using it for functions what use function(s) as argument(s) (map, filter et consortes...). After all it's something like lambda, but: * more beautiful (IMHO), * not limited to expression. Regards, -- Jan Kaliszewski <zuo@chopin.edu.pl>
"Jan Kaliszewski" <zuo@chopin.edu.pl> writes:
And what about that:
class SomeClass: ...
def summarize_proust(self, objects, args, method_name) "Summarize Proust in 15 seconds"
return map(action, objects, args, alts) where:
def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result
alts = map(weirdtrans, objects) where def weirdtrans(obj): # some transformations not very # useful in more general context
You then make an invalid comparison of this versus making ‘weirdtrans’ an unnecessary instance method. But the correct equivalent of your example above is to make another local function:: class SomeClass: # … def summarize_proust(self, objects, args, method_name): """ Summarize Proust in 15 seconds. """ def weirdtrans(obj): # some transformations not very # useful in more general context alts = map(weirdtrans, objects) def action(obj, arg, alt): result = getattr(item, method_name)(arg) or alt obj.set_last_result(result) return result return map(action, objects, args, alts) That seems more straightforward to me. Certainly I don't find the one with ‘where’ any clearer. -- \ “Holy unrefillable prescriptions, Batman!” —Robin | `\ | _o__) | Ben Finney
Gerald Britton <gerald.britton@gmail.com> writes:
Anyway, I like the "where" idea not because of real or imagined performance gains, but because of its cleanness when expressing problems.
(Speaking of cwclean expression, please don't top-post; instead, remove irrelevant quoted material and respond beneath each remaining point <URL:http://en.wikipedia.org/wiki/Posting_style#Inline_replying>.)
A big use for me would be in list comprehensions. In one project I work on, I see things like:
for i in [item in self.someobject.get_generator() if self.important_test(item)]
and other really long object references.
which I would like to write:
mylist = [item in f if g(item)] where: f = self.someobject.get_generator() g = self.important_test
I presume these two are supposed to be equivalent (and syntactically valid), so I'll take it the first one should be something like:: mylist = [item for item in self.someobject.get_generator() if self.important_test(item)]
To my eyes, the first is harder to read than the second one.
That's largely attributable to the fact that you've got one alternative all on a single line, and the other broken into more easily-readable lines. I don't think the ‘where’ syntax is much help there. I would write the single-statement version as:: mylist = [ item for item in self.someobject.get_generator() if self.important_test(item)] which makes it far more readable. I argue that this does away with pretty much any justification for your use case above. -- \ “… a Microsoft Certified System Engineer is to information | `\ technology as a McDonalds Certified Food Specialist is to the | _o__) culinary arts.” —Michael Bacarella | Ben Finney
On Sat, 18 Jul 2009 06:28:40 am Gerald Britton wrote:
I often time my code and often find that removing function calls "where" possible actually makes a measurable difference. This is especially the case with little getters and setters in class defs.
Perhaps you shouldn't be writing Java code with lots of getters and setters then *wink* Seriously, I've very glad to hear you're measuring before optimising, but a "measurable difference" is not necessarily a significant difference. Anyway, we're not here to discuss the pros and cons of optimization, so back to the top on hand:
Anyway, I like the "where" idea not because of real or imagined performance gains, but because of its cleanness when expressing problems.
I just don't see this cleanness. It seems to me that the "where" construct makes writing code easier than reading code. It is a top-down construct: first you come up with some abstract expression: element = w + x.y - f(z) where: and then you need to think about how to implement the construct: w = 2 x = thingy() z = 4 That roughly models the process of writing code in the first place, so I can see the attraction. But it doesn't model the process of *reading* code: "Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any of those names before, are they globals? Ah, there's a 'where' statement, better pop that expression into short-term memory while I read the block and find out what they are..." In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process: def parrot(): x = 1 y = 2 z = 3 result = x+y+z return result A complication: if the function you are reading relies on globals, including functions, then you may need to jump out of the function to discover what it is. But if you already know what the function does, or can infer it from the name, then it doesn't disrupt the sequential reading. But the where clause introduces *look-ahead* to the process: def parrot(): x = 1 result = x+y+z where: # look-ahead y = 2 z = 3 return result This is not a heavy burden if you are limited to a few simple names, but as soon as you re-use pre-existing names in the 'where' scope, or increase its complexity, readability suffers greatly. def harder(): w = 5 z = 2 result = w + x.y - f(z) where: w = 2 class K: y = 3 x = K() def f(a): return a**2 + a z = 4 return result Look-ahead is simply harder on the reader than reading sequentially. You can abuse any tool, write spaghetti code in any language, but some idioms encourage misuse, and in my opinion this is one of them.
A big use for me would be in list comprehensions. In one project I work on, I see things like:
for i in [item in self.someobject.get_generator() if self.important_test(item)]
and other really long object references.
I don't see anything objectionable in that. It's a tad on the long side, but not excessively.
which I would like to write:
mylist = [item in f if g(item)] where: f = self.someobject.get_generator() g = self.important_test
I don't think much of your naming convention. Surely g should be used for the generator, and f for the function, instead of the other way around?
To my eyes, the first is harder to read than the second one. Of course I can do this:
f = self.someobject.get_generator() g = self.important_test mylist = [item in f if g(item)]:
but then "f" and "g" pollute the calling context's namespace.
Holy cow! How many variable names do you have in a single function that you have to worry about that??? Honestly, I think this entire proposal is a pessimation: you're proposing to significantly complicate the language and negatively impact the readability of code in order to avoid polluting the namespace of a function? Of course, then people will start worrying about "polluting the namespace" of the where-block, and start doing this: result = x + y + z where: x = 1 y = 2 z = a*b - c where: a = 5 b = 6 c = d*e where: d = 3 e = 4 -- Steven D'Aprano
[quote-trimming ahead] On Sat, Jul 18, 2009, Steven D'Aprano wrote:
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
But the where clause introduces *look-ahead* to the process:
Look-ahead is simply harder on the reader than reading sequentially.
You can abuse any tool, write spaghetti code in any language, but some idioms encourage misuse, and in my opinion this is one of them.
This is the clearest indictment yet -- I've mostly been ignoring this thread figuring it'll die a natural death on its own, but this makes me add my own -1 to the ``where`` clause. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair
Steven D'Aprano wrote:
It seems to me that the "where" construct makes writing code easier than reading code. It is a top-down construct: first you come up with some abstract expression:
element = w + x.y - f(z) where:
and then you need to think about how to implement the construct:
w = 2 x = thingy() z = 4
That roughly models the process of writing code in the first place, so I can see the attraction. But it doesn't model the process of *reading* code:
"Hmmm, element is equal to w + x.y - f(z), okay, but I haven't seen any of those names before, are they globals? Ah, there's a 'where' statement, better pop that expression into short-term memory while I read the block and find out what they are..."
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
This objection seems to be a very convincing one. At the very least, I think we should give up on the idea of having the "where" block use a different scope, since that seems to cause more problems than it solves. For me, the original reason why where clauses looked appealing is that I think there are some very limited cases in which the code reads better out of order. My canonical example for this is "new_list = sorted(old_list, key=...)" It just seems like sometimes you don't want to hear about the key func until after you know that you're going to sort a list. A simple "where" (or even "do … where" which might read a little better) statement would solve this problem by basically just letting you one line out of order. do new_list = sorted(old_list, key=key) where: def key(item): do_data_normalizing_stuff… I also think that most decorator definitions would look better with the return at the top rather than the bottom: def my_debug_decorator(f): do return inner where: def inner(*args, **kwargs): print(args, kwargs) return f(*args, **kwargs) Yes, this nests a level deeper than usual (flat is better than nested), but I think it's a lot less confusing than the usual form of decorators with the return at the end where your eye doesn't know where to rest. On the other hand, it's not that hard to make a decorator decorator to get rid of the weird return clause, and having any where statement at all invites the abuse of nested wheres: do x += y where: do y = z where: do z = x where: x = 2 print(x) # 4 Is the temptation for abuse here so strong that it will out weigh the potential for readability gains? Does this construct violate TOOWTDI? My answer is a definite… maybe. So for now, I think "where" is not ready for prime time. My 2 cents, -- Carl Johnson
Steven D'Aprano wrote:
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
Here you're assuming that the order of *execution* of the code is necessarily the best order for *reading* it in order to understand what it does. In general I think that's mostly false. When encountering a program for the first time, one tends to look for the "main" function first, to get an overall idea of what's being done, then recursively work one's way down into lower level functions. Python supports this on a large scale by allowing functions and classes to be written pretty much in any order, regardless of the order of execution. There's not much support for it on a small scale, though. There is a little. List comprehensions are one example, where you get to see the high-level intent first -- make a list of stuff -- and then you're shown the lower-level details. Another one that's easy to overlook is the assignment statement. Nobody seems to be bothered by the fact that the *right* hand side has to be evaluated before assigning to the left hand side. In a sense, the where-block is in part an attempt to extend the assignment statement so that the "right hand side" can span more than one statement. -- Greg
Greg Ewing <greg.ewing@canterbury.ac.nz> writes:
In a sense, the where-block is in part an attempt to extend the assignment statement so that the "right hand side" can span more than one statement.
That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement expressions are verboten. -- \ “Pinky, are you pondering what I'm pondering?” “I think so, | `\ Brain, but why does a forklift have to be so big if all it does | _o__) is lift forks?” —_Pinky and The Brain_ | Ben Finney
Ben Finney <ben+python@benfinney.id.au> wrote:
Greg Ewing <greg.ewing@canterbury.ac.nz> writes:
In a sense, the where-block is in part an attempt to extend the assignment statement so that the "right hand side" can span more than one statement.
That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement expressions are verboten.
But where-statement is not an expression at all -- yet it's a sequence of statements. And even in that PEP you can read that a new syntax for anonymous functions would be welcone: "At one point lambda was slated for removal in Python 3000. Unfortunately no one was able to come up with a better way of providing anonymous functions. And so lambda is here to stay." IMHO 'where' would be better than lambda in many contexts, especially if there is a need for function as argument, e.g. for filter(), map(), sort(key=...) etc., and also for some list comprehensions and generator expressions. Regards, -- Jan Kaliszewski <zuo@chopin.edu.pl>
Ben Finney wrote:
That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement expressions are verboten.
But it's not a multi-statement expression. It's a statement with a suite, and it works entirely within the existing framework of the grammar. -- Greg
On Sat, Jul 18, 2009 at 5:06 PM, Greg Ewing<greg.ewing@canterbury.ac.nz> wrote:
Ben Finney wrote:
That pretty much spells its doom, then. The BDFL is quite firm (as demonstrated in, e.g., the pronouncement against multi-line lambda <URL:http://www.python.org/dev/peps/pep-3099/>) that multi-statement expressions are verboten.
But it's not a multi-statement expression. It's a statement with a suite, and it works entirely within the existing framework of the grammar.
The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-). However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence. -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum <guido@python.org> wrote:
The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-).
However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence.
I believe it's at the other 'end for stick' of development process and doesn't hurt 3.1 at all. It could appear at the earliest in 3.2 or 3.3 in __future__... but I think it doesn't mean we should freeze working on new ideas, even if they have chance to be released not earlier that in three or five years. Mike Meyer <mwm-keyword-python.b4bdba@mired.org> wrote:
but the BDFL said he sort of liked it, except for the name - and that we should be shaking out 3.X instead of extending the language. So, just to get it on record, how about a B&D version of the where statement? Let's call it the "with" clause.
But we already have 'with' in the language, doing completely different things. I propose 'using'. Regards, zuo -- Jan Kaliszewski <zuo@chopin.edu.pl>
On Sun, Jul 19, 2009 at 11:29 AM, Jan Kaliszewski<zuo@chopin.edu.pl> wrote:
Guido van Rossum <guido@python.org> wrote:
The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-).
However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence.
I believe it's at the other 'end for stick' of development process and doesn't hurt 3.1 at all.
Actually, adding too many new features post 3.1 *could* hurt 3.1 -- it could slow adoption because people might decide to wait for 3.2 which they expect to be even better. Plus the implementation of 3.2 features uses up energy that would be better spent elsewhere (the Python community has a chronic shortage of hands to help with menial task like porting 3rd party code to 3.1). Another argument against new features is that the people who have the hardest time moving to 3.1 are those who are most bothered by change in general. If they view the language as "moving too fast" they might decide to move to a language that moves more slowly, or just stick with Python 2.4, which is nearly as bad. A more cynical view (the like of which I've heard expressed about Perl 6) would be that python-ideas is where we keep those folks occupied who love to argue about new features but are unlikely to contribute anything... :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/)
Guido van Rossum wrote:
On Sun, Jul 19, 2009 at 11:29 AM, Jan Kaliszewski<zuo@chopin.edu.pl> wrote:
Guido van Rossum <guido@python.org> wrote:
The 'where' proposal actually looks nice to me. (Maybe I've been studying functional languages too much lately :-). My biggest problem with this concrete proposal is that 'where' means something completely different in SQL (which I've also studied too much lately :-).
However, I think we should focus on keeping the language stable rather than keep tinkering with it. Let's help 3rd party developers port their work to 3.1 rather than planning 3.1's obsolescence. I believe it's at the other 'end for stick' of development process and doesn't hurt 3.1 at all.
Actually, adding too many new features post 3.1 *could* hurt 3.1 -- it could slow adoption because people might decide to wait for 3.2 which they expect to be even better. Plus the implementation of 3.2 features uses up energy that would be better spent elsewhere (the Python community has a chronic shortage of hands to help with menial task like porting 3rd party code to 3.1).
Another argument against new features is that the people who have the hardest time moving to 3.1 are those who are most bothered by change in general. If they view the language as "moving too fast" they might decide to move to a language that moves more slowly, or just stick with Python 2.4, which is nearly as bad.
A more cynical view (the like of which I've heard expressed about Perl 6) would be that python-ideas is where we keep those folks occupied who love to argue about new features but are unlikely to contribute anything... :-)
Or it's where the bikeshedding gets done; if an idea makes out of python-ideas then any critic of the new feature can be pointed to the relevant thread and told "already discussed, see!". :-)
On Sat, 18 Jul 2009 02:20:43 pm Greg Ewing wrote:
Steven D'Aprano wrote:
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
Here you're assuming that the order of *execution* of the code is necessarily the best order for *reading* it in order to understand what it does.
In general I think that's mostly false. When encountering a program for the first time, one tends to look for the "main" function first, to get an overall idea of what's being done, then recursively work one's way down into lower level functions.
I was explicitly talking about reading a *function*, not the entire program. In general, one does not try to keep the entire program in one's brain at once (unless it's a trivial program) -- one swaps individual routines in and out as required: "Hmm, let's see how the main function works... okay, first it grabs input from the user, now that I know that, I can pop the details from short-term memory and just treat it as a 'get input' black box. Next it creates a Pager object, what does that do... oh, it creates pages, duh, now I can forget the details and treat it as a black box as well... " (At least I do this.) But for understanding a single function, you do need to keep the whole thing in your mind at once: you can't treat it as a black box, because that defeats the purpose of trying to *understand* it. (If you want a black box, just read the doc string and function signature and you're done.) That's one of the reasons why giant monolithic functions are so difficult to understand, even if all they do is one thing. It's also why spaghetti code within a single function is so harmful: it defeats the reader's ability to compartmentalise functionality into easily understandable chunks.
Python supports this on a large scale by allowing functions and classes to be written pretty much in any order, regardless of the order of execution.
Yes, and so it should -- a certain amount of "spaghetti" is unavoidable in any non-trivial program, but so long as it's kept under control, it's not too bad. But a single function is different.
There's not much support for it on a small scale, though. There is a little. List comprehensions are one example, where you get to see the high-level intent first -- make a list of stuff -- and then you're shown the lower-level details.
That's hardly different from a for loop though.
Another one that's easy to overlook is the assignment statement. Nobody seems to be bothered by the fact that the *right* hand side has to be evaluated before assigning to the left hand side.
I've often wondered whether coding would be simpler (at least for beginners) if we wrote assignment left-to-right like this: 2*x + 1 => y instead of y = 2*x + 1 That might be a nice experiment for a teaching language some day: does left-to-right assignment reduce or eliminate the confusion that beginners experience over assignment? In fact, the more I think about it, the more I like the idea. Oh yes... Hypertalk does that, only very verbosely: put 2*x + 1 into y And doesn't Cobol do something similar?
In a sense, the where-block is in part an attempt to extend the assignment statement so that the "right hand side" can span more than one statement.
Yes, I get that. I understand the rationale for it, I just think it's a bad idea. -- Steven D'Aprano
[OT] Steven D'Aprano <steve@pearwood.info> wrote:
I've often wondered whether coding would be simpler (at least for beginners) if we wrote assignment left-to-right like this:
2*x + 1 => y
instead of
y = 2*x + 1
That might be a nice experiment for a teaching language some day: does left-to-right assignment reduce or eliminate the confusion that beginners experience over assignment?
In fact, the more I think about it, the more I like the idea.
Oh yes... Hypertalk does that, only very verbosely:
put 2*x + 1 into y
And doesn't Cobol do something similar?
Chuck does it, and just with '=>' :) (see: http://chuck.cs.princeton.edu/ ) *j
About return, yield and their ilk in where clauses: don't do it! Seriously, a where clause should be restricted to providing definitions for the things in the statement for which the where clause is specified. Today, we can't do a return (with argument) in a generator function, which makes sense. A where clause should not allow returns or yields or in general actions that explicitly transfer control beyond the statement.
Gerald Britton <gerald.britton@gmail.com> wrote:
About return, yield and their ilk in where clauses:
don't do it!
Seriously, a where clause should be restricted to providing definitions for the things in the statement for which the where clause is specified. Today, we can't do a return (with argument) in a generator function, which makes sense. A where clause should not allow returns or yields or in general actions that explicitly transfer control beyond the statement.
+1 -- Jan Kaliszewski <zuo@chopin.edu.pl>
Steven D'Aprano wrote:
But the where clause introduces *look-ahead* to the process:
def parrot(): x = 1 result = x+y+z where: # look-ahead y = 2 z = 3 return result
I think this example is too rarified to give much idea of how readable a where-statement might be. In real life you wouldn't use arbitrary names like x, y, z. You'd choose something to make the expression meaningful in its own right, so you can get the gist of what's going on just from reading the first line. Then you only need to look inside the where block if you want to know the fine details. Also, I don't think that "look ahead" is necessarily harmful unless the stuff ahead has side effects. We seem to be happy with "look somewhere else" when calling a function, whose definition could appear textually before or after the point of call. -- Greg
On Sat, 18 Jul 2009 15:33:59 +1200 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Steven D'Aprano wrote:
But the where clause introduces *look-ahead* to the process:
def parrot(): x = 1 result = x+y+z where: # look-ahead y = 2 z = 3 return result
I think this example is too rarified to give much idea of how readable a where-statement might be.
I, on the other hand, am convinced that this example is dead on. By requiring the definition to be contextually close to the use, you'll encourage people to use shorter, less meaningful names. Or can you honestly say that you use names of the same length and carrying as much meaning for local variables as you do for globals, attributes and parameters?
In real life you wouldn't use arbitrary names like x, y, z. You'd choose something to make the expression meaningful in its own right, so you can get the gist of what's going on just from reading the first line.
What where you're pointing your pronouns! I know you don't speak for me, and I'm pretty sure you don't speak for Steven (but I don't either, and he may agree with you). I'd considered where to be promising, assuming that the problems it has could be worked through. But Steven's observations are deadly. Not that the original impulse wasn't good - trying to find things that scratch the itches that cause repeated "function definition in an expression" proposals is almost certainly worthwhile. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On Fri, Jul 17, 2009 at 7:39 PM, Steven D'Aprano <steve@pearwood.info>wrote:
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
Normally, that's true, but that's not true when defining a sub-function. Consider the following: def foo(items): def compute_sort_value(item): # compute value based on item's properties return value # a bunch of other code is here items.sort(compute_sort_value) Above, the body of compute_sort_value appears long before it is executed. Now consider: def foo(items): # a bunch of other code is here items.sort(key=mean) where: def compute_sort_value(item): return value Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort(). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On Sat, 18 Jul 2009 10:45:44 pm Daniel Stutzbach wrote:
On Fri, Jul 17, 2009 at 7:39 PM, Steven D'Aprano <steve@pearwood.info>wrote:
In a real sense, the proposed 'where' block breaks the flow of reading code. Normally, reading a function proceeds in an orderly fashion from the start of the function to the end in (mostly) sequential order, a bottom-up process:
Normally, that's true, but that's not true when defining a sub-function. Consider the following:
def foo(items): def compute_sort_value(item): # compute value based on item's properties return value # a bunch of other code is here items.sort(compute_sort_value)
Above, the body of compute_sort_value appears long before it is executed.
Right. That's no different from this case: def bar(items): c_s_v = list( # comment goes here 'csv') # a bunch of other code is here items.extend(c_s_v) def is an executable statement which creates a function; list() is an executable function (technically a type) which creates an list. In your example, the inner function compute_sort_value is created when foo is executed. Python functions are first class objects, and so you can create functions and pass them to functions without necessarily calling the __call__ method. In my example, the inner list c_s_v is created when bar is executed. Lists are first class objects, and so you can create lists and pass them to functions without necessarily calling the __getitem__ method. In both functions, the objects compute_sort_value and c_s_v are created before they are used.
Now consider:
def foo(items): # a bunch of other code is here items.sort(key=mean) where: def compute_sort_value(item): return value
Presumably you meant to write: items.sort(key=compute_sort_value) where:
Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort().
No it doesn't. The function definition appears outside the call to sort(). For the function definition to appear inside the call to sort, you'd need to write something like: items.sort(key=lambda item: value) In this case, the lambda only occurs inside the call to sort() -- it's inaccessible to anything else (excluding trickery inside of sort() itself). By comparison, the compute_sort_value() function exists inside the where-block, and therefore is accessible to everything else inside that block: items.sort(key=compute_sort_value) where: def compute_sort_value(item): return value print compute_sort_value('testing testing 1 2 3') -- Steven D'Aprano
items.sort(key=compute_sort_value) where:
Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort().
No it doesn't. The function definition appears outside the call to sort(). For the function definition to appear inside the call to sort, you'd need to write something like
Yes, it does appear within the call. The call is more than just the parameters in parentheses. It begins with "items.sort" and ends (in this case) with then end of the "where" clause
items.sort(key=lambda item: value)
The idea behind "where" clauses is not to replace lambdas. In fact, they are useful together: items.sort(key=mysort) where: mysort = sort_key_function_factory( with, many, arguments) which to many is preferable over: items.sort(key=lambda item:sort_key_function_factory(with, many, arguments)) Even though you can split it: items.sort(key=lambda item: sort_key_function_factory(with, many, arguments) )
In this case, the lambda only occurs inside the call to sort() -- it's inaccessible to anything else (excluding trickery inside of sort() itself). By comparison, the compute_sort_value() function exists inside the where-block, and therefore is accessible to everything else inside that block:
items.sort(key=compute_sort_value) where: def compute_sort_value(item): return value print compute_sort_value('testing testing 1 2 3')
That would be useful for debugging. Really though, the whole thing comes down to a matter of taste and visual presentation. Some will find "where" completely intuitive and a natural way to work. Others will loathe it (though as has been pointed out, it really makes using Haskell functions easier to write and to read.) On the whole though, it's a +1 for me
On Sun, 19 Jul 2009 06:11:57 am Gerald Britton wrote:
items.sort(key=compute_sort_value) where:
Now, the body of compute_sort_value appears exactly where it is executed: within the call to items.sort().
No it doesn't. The function definition appears outside the call to sort(). For the function definition to appear inside the call to sort, you'd need to write something like
Yes, it does appear within the call. The call is more than just the parameters in parentheses. It begins with "items.sort" and ends (in this case) with then end of the "where" clause
That's a definition of "within" that I'm not really happy with. And I think it's wrong. Consider the timing of calls: import time class MyList(list): def sort(self, *args, **kwargs): print "Call to sort() started at %s seconds" % time.time() super(MyList, self).sort(*args, **kwargs) print "Call to sort() ended at %s seconds" % time.time() items = MyList([-9, 7, -4, 2, -1, 0, 3]) items.sort(key=func) where: print "Enter the where-block at %s seconds" % time.time() time.sleep(555) def func(item): return abs(item) You need to create func before you can pass it as an argument to sort(), so I would expect the above to print something like: Enter the where-block at 1234560000.23 seconds Call to sort() started at 1234560555.24 seconds Call to sort() ended at 1234560555.25 seconds To my mind, to say that the contents of the where-block occur "within" the call to sort, it would imply that you get output like this: Call to sort() started at 1234560000.23 seconds Enter the where-block at 1234560000.24 seconds Call to sort() ended at 1234560555.25 seconds
items.sort(key=lambda item: value)
The idea behind "where" clauses is not to replace lambdas. In fact, they are useful together:
items.sort(key=mysort) where: mysort = sort_key_function_factory( with, many, arguments)
If you're going to say that where and lambda are useful together, shouldn't your example actually show where and lambda together?
which to many is preferable over:
items.sort(key=lambda item:sort_key_function_factory(with, many, arguments))
That's equivalent to: def f(item): function = sort_key_function_factory(with, many, arguments) return function items.sort(key=f) which almost certainly is not what you what. I think what you meant was: items.sort(key=lambda item: sort_key_function_factory(with, many, arguments)(item) ) # note that you call the function returned by the factory which at least works, but it adds a pointless layer of indirection. What you should have is: items.sort(key=sort_key_function_factory(with, many, arguments)) which of course is simpler than your earlier example with the where-block. -- Steven D'Aprano
Steven D'Aprano wrote:
That's a definition of "within" that I'm not really happy with. And I think it's wrong. Consider the timing of calls:
To my mind, to say that the contents of the where-block occur "within" the call to sort, it would imply that you get output like this:
Call to sort() started at 1234560000.23 seconds Enter the where-block at 1234560000.24 seconds Call to sort() ended at 1234560555.25 seconds
By that definition though, a lambda isn't compiled "within" the sort expression. First the lambda is compiled then it is passed as an argument to sort. But you earlier wrote that,
For the function definition to appear inside the call to sort, you'd need to write something like:
items.sort(key=lambda item: value)
But that doesn't follow at all. Items.sort isn't called until there are arguments that can be passed to it. The timing of this would be the exact same as a where block:
import time class MyList(list): ... def sort(self, *args, **kwargs): ... print("Call to sort() started at %s seconds" % time.time()) ... super(MyList, self).sort(*args, **kwargs) ... print("Call to sort() ended at %s seconds" % time.time()) ... items = MyList([-9, 7, -4, 2, -1, 0, 3])
def do_stuff(): ... print("Enter the fake where-block at %s seconds" % time.time()) ... time.sleep(10) ... items.sort(key=lambda item, default=do_stuff(): abs(item)) Enter the where-block at 1247970018.89 seconds Call to sort() started at 1247970028.89 seconds Call to sort() ended at 1247970028.89 seconds
-- Carl
On Sat, Jul 18, 2009 at 9:08 PM, Steven D'Aprano <steve@pearwood.info>wrote:
Yes, it does appear within the call. The call is more than just the parameters in parentheses. It begins with "items.sort" and ends (in this case) with then end of the "where" clause
That's a definition of "within" that I'm not really happy with. And I think it's wrong. Consider the timing of calls:
I think we're arguing semantics there. There are some reasonable definitions of "within" where the "where" statement isn't within the call, and other reasonable definitions of "within" where it after the call. You've already given examples of the later. Here's an example of the former: for i in range(5): x.append(some_function(i)) I think we can agree that the body of the "for" loop is within the "for" statement? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
I wrote this, then pretty much rejected this due to the "upside-down" nature of where, but the BDFL said he sort of liked it, except for the name - and that we should be shaking out 3.X instead of extending the language. So, just to get it on record, how about a B&D version of the where statement? Let's call it the "with" clause. The grammar extension (based on the 2.6 grammer) would be: stmt_with_with: small_stmt 'with' name_list':' bind_suite bind_suite = bind_stmt | NEWLINE INDENT bind_stmt+ DEDENT bind_stmt = expr_stmt | funcdef | classdef name_list = NAME (',' NAME)* The semantics look like this (modulo the non-hygenic nature of the macro conversion): old_locals = locals() old_names = set(old_locals) bind_suite new_names = old_names.union(name_list) for key in locals: if key not in new_names: del key small_stmt for key in name_list: if key in old_locals: eval '%s = %s' % (key, old_locals[key]) else: del key The bind_suite is effectively run in a new scope, as the current scope won't be affected by any bindings in it. In particular, global and friends can't be used in bind_suite. The statement that the with clause is attached to is then executed in the enclosing scope - as that's where any bindings will happen, with the variables in name_list, and no others, added to it. After that, we restore the existence/values of the variables in name_list to their old value. Yes, this means if you do something like: x = a + b with x, a, b: a = 2 b = 3 then the value of x is going to get lost afterwards. That's a bug in your code. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
MRAB wrote:
"do" might be a sufficiently generic word to cover all the cases.
if a == b where: ... do: ...
It's probably okay for 'if', but to me it doesn't read right with 'def' or 'class'. But in any case, I'm -0.9 on allowing 'where' on arbitrary statements in the first place. Seems like feature creep on a massive scale to me. (Feature stampede?) -- Greg
2009/7/15 Carl Johnson <cmjohnson.mailinglist@gmail.com>:
My suggestion for future would be language extenders is that they try to solve a broader problem than just the "how do I make callbacks more convenient" problem. Just solving that one problem by itself seems unlikely to make it into the language, since it is pure "sugar" and Pythoneers have a well justified wariness toward anonymous functions (why not just name them and be done with it?). Perhaps someone should take a look at the original "with" proposal, or some of the various "do" and "block" proposals. One idea I think about sometimes is having a more convenient way to do something along the lines of metaclass.__prepare__ but for functions…
I agree entirely. There's certainly *something* here worthy of consideration, otherwise it wouldn't keep coming up. But there's probably a more general construct (yes, I know, "fully general anonymous functions" - maybe there's something *else*...?) lurking underneath all of the proposals, and it's worth waiting until someone discovers it. The important principle that George quoted: 2009/7/15 George Sakkis <george.sakkis@gmail.com>:
Thanks, this was it. The important part is Guido's quote "any solution that embeds an indentation-based block in the middle of an expression is unacceptable", so it's basically a design judgement, not an implementation constraint.
is key here - keep blocks and expressions distinct. Future proposals should keep that in mind. Maybe there's value in having a "language design principles" PEP where pronouncements like this could be recorded, for the benefit of people proposing new features? Paul.
On Wed, Jul 15, 2009 at 4:27 AM, Paul Moore<p.f.moore@gmail.com> wrote:
The important principle that George quoted:
2009/7/15 George Sakkis <george.sakkis@gmail.com>:
Thanks, this was it. The important part is Guido's quote "any solution that embeds an indentation-based block in the middle of an expression is unacceptable", so it's basically a design judgement, not an implementation constraint.
is key here - keep blocks and expressions distinct. Future proposals should keep that in mind.
I think you may be misinterpreting - note Guido's use of the word "middle". That is precisely why I proposed allowing blocks only at the _end_ of expressions, and not in the middle. OTOH, I may just be projecting the interpretation of that statement that suits me best. :) To answer some of the other points that have been raised: Steven:
Did I understand correctly that you prohibited using `if`, `while` and other block statements inside the do block? No, you misunderstood. What you are thinking of is the fact that the test expression in an if block, or a while block, cannot have an appended block. A block is really just syntactic sugar for a def - it lacks only a name, and the ability to be decorated. It can have a docstring, it can be a generator, etc.
Steven:
Personally, the only time I've ever missed the ability to create multi-line lambdas was when I had a series of code blocks like this:
try: BLOCK except exception, e: process_error(e)
where the BLOCK was different in each one, and I really wanted to factor out the common code into a function "process" and pass each BLOCK as an argument:
while condition: process(BLOCK1) process(BLOCK2) if something: process(BLOCK3)
sort of thing.
Do you mean something like this? def safely(body): try: body() except exception, e: process_error(e) while condition: safely() do: BLOCK1 safely() do: BLOCK2 if something: safely() do: BLOCK3 I don't claim that a pattern like that will work in all cases, because the body of the blocks have their own local scope. If you need to set a variable in the enclosing scope, it might not do what you want (but we do have "nonlocal" now, so that helps). Carl:
That said, I have to insist that what you've proposed as
f() do: BLOCK
be written instead as
f(&) do: BLOCK
Just magically inserting the block as the last argument to a call is something I hate about Ruby
Sure, that's OK with me. I also expected the argument that the empty parentheses after "do" should not be optional, for symmetry with def, and that's fine with me to. I was just trying to keep the syntax as "lightweight" as possible for the simplest cases. Stephen:
Some people feel that giving names to a cognitive chunk is an essential part of the process of chunking. They are made uncomfortable, and often confused, by code which is named only by its own source (aka "blocks"). Others don't feel that way. The former oppose this kind of proposal strongly, the latter support it more or less actively.
Agreed, and I do have a slight nagging worry that blocks could be abused, in that people could use them in cases where a named function would be better (and I agree that there will always be many such cases). But then I remember that "we're all adults here", and that in practice people can just as easily, and often do, give useless names to small functions, and then I don't worry about it so much anymore. Carl:
If it's the lack of function names and docstrings
Python 2.7a0 (trunk, Jul 12 2009, 07:54:12) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
def f(b): return b.__doc__ ... f() do: ... "Look, ma, I'm a docstring!" ... "Look, ma, I'm a docstring!"
Overall, I think we're at the point where most of us agree that this is a very subjective matter, and it comes down to taste. Guido has a well-recorded history of saying (paraphrasing) "just write a def with a name and quit whining!", so I suppose the odds for this proposal are not great. But I have certainly enjoyed the discussion, so thanks to everyone who has participated. Chris Perkins
On Wed, Jul 15, 2009, Paul Moore wrote:
Maybe there's value in having a "language design principles" PEP where pronouncements like this could be recorded, for the benefit of people proposing new features?
Sure, although PEP 20 seems like an obvious spot. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair
On Wed, Jul 15, 2009 at 8:30 AM, Aahz<aahz@pythoncraft.com> wrote:
On Wed, Jul 15, 2009, Paul Moore wrote:
Maybe there's value in having a "language design principles" PEP where pronouncements like this could be recorded, for the benefit of people proposing new features?
Sure, although PEP 20 seems like an obvious spot.
Also, PEP 3099, although it's more about prohibitions than principles: http://www.python.org/dev/peps/pep-3099/ Cheers, Chris -- http://blog.rebertia.com
On Wed, 15 Jul 2009 02:56:38 am Chris Perkins wrote:
For me, avoiding thinking of a name was never the primary motivation - putting things in the "right order" is. But I do believe that there are situations where the context that a block of code sits in tells far more about its purpose that a name ever could - in those circumstances, a forced function name can be useless at best, and distracting clutter at worst. For example:
y = sorted(x, key=&) do(item): name = item.split('-')[1] return name.upper()
I think that forcing that two-line block of code to have a name would serve no purpose - it's abundantly clear what it does.
I'm afraid I'm with Paul Moore on this one -- it's not clear to me either. The key=& seems like a mysterious Perlism -- what's the & operator doing in there? (Admittedly, it *might* be nice to be able to write things like reduce(&, alist) instead of reduce(operator.and_, alist). But that's another story.) "do(item):" confuses me, because it looks like the start of a do loop, but it isn't. None of this is to say that I couldn't learn the proposed syntax, but in my opinion it doesn't mesh well with existing Python constructs. Did I understand correctly that you prohibited using `if`, `while` and other block statements inside the do block? If so, then to my mind the proposal is useless -- it's neither an expression, like the body of lambda, nor a block, like the body of a function, but a freakish chimera.
I also think that the current state of affairs encourages poorer readability - I mean, admit it, today you would probably write that snippet like this:
y = sorted(x, key=lambda item: item.split('-')[0].upper())
which to me loses clarity both by being crammed into one line, and by not being able to create the local variable "name", which tells you quite a bit about why the the item is being chopped up the way it is.
If I were doing that operation more than once, I'd make a function: def get_canonical_name(item): return item.split('-')[0].upper() and I'd give it a docstring and doctests. The function get_canonical_name tells you *much* more about why the item is processed the way it is than a mere local variable "name". You'll note that I wouldn't bother using a local -- this is short enough, and simple enough, that I don't need it. But if I only did it once, then I see nothing wrong with the lambda version. If you're worried about it being "crammed" into one line: y = sorted(x, # use the canonical name as the sort key key=lambda item: item.split('-')[0].upper() ) works for me.
Seriously, I guess this is where we differ - I think that improved readability _is_ a sufficient goal unto itself. The only question is whether other people (OK, one other person in particular) thinks it improves readability, and if so, then by how much.
Personally, the only time I've ever missed the ability to create multi-line lambdas was when I had a series of code blocks like this: try: BLOCK except exception, e: process_error(e) where the BLOCK was different in each one, and I really wanted to factor out the common code into a function "process" and pass each BLOCK as an argument: while condition: process(BLOCK1) process(BLOCK2) if something: process(BLOCK3) sort of thing. To my mind, multi-line lambdas are really only useful for the ability to factor out common code, not to avoid defining a function or spreading lambdas out over multiple lines. -- Steven D'Aprano
Steven D'Aprano wrote:
The key=& seems like a mysterious Perlism -- what's the & operator doing in there? (Admittedly, it *might* be nice to be able to write things like reduce(&, alist) instead of reduce(operator.and_, alist). But that's another story.) "do(item):" confuses me, because it looks like the start of a do loop, but it isn't.
The thing about this debate that confuses me is why was the @ decorator ever approved in the first place? It's line-noise, and worse, it's line-noise with a meaning completely different from any other language, ever. On top of that if you do help(), you get nothing [Ed: in Python <2.6. Good job, whoever fixed this!], and Googling is worthless. When I first learned Python, I was very confused by decorators--in part because I didn't know "decorator" was the term to Google for when trying to figure out the meaning of "@"! And yet, @ was approved, in spite of its worthlessness (it literally adds no power to the language, just sugar) and strangeness because of the perceived benefits to readability. And in my opinion, there are similar cases here, where readability is improved by having the definition of a function come after the expression to which the function will be passed. Yet, there's a lot of resistance to this, and I'm not entirely sure why. If it's just a problem with using "&" and "do" other line-noise and keywords can be proposed to substitute for it. $ has no meaning in Python, for example. If it's the lack of function names and docstrings, there's no reason these lambda-like-thingies can't have support for names and docstrings added (as in my old proposal, for example). But I feel like there's some deeper reason people don't think this is a readability win. What is it? -- Carl
On Tue, 14 Jul 2009 14:38:34 -1000 Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote:
Steven D'Aprano wrote:
The key=& seems like a mysterious Perlism -- what's the & operator doing in there? (Admittedly, it *might* be nice to be able to write things like reduce(&, alist) instead of reduce(operator.and_, alist). But that's another story.) "do(item):" confuses me, because it looks like the start of a do loop, but it isn't.
The thing about this debate that confuses me is why was the @ decorator ever approved in the first place? It's line-noise, and worse, it's line-noise with a meaning completely different from any other language, ever.
You just stumbled on it. What other language, ever, had something like decorators? It's a magic syntax that performs magic on the following function definition. An unusual symbol in an unusual location is a clue that something unusual is going on. For a more thorough discussion of the reason for the adopted syntax, read the PEP: http://www.python.org/dev/peps/pep-0318/. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
On 7/14/09, Carl Johnson <cmjohnson.mailinglist@gmail.com> wrote:
Steven D'Aprano wrote:
The thing about this debate that confuses me is why was the @ decorator ever approved in the first place?
It wasn't, for quite a while. I believe it was proposed for 2.2. I'm sure it was proposed for 2.3. Applying it to classes as well as functions was in the working patch -- but waited until another version just in case.
And yet, @ was approved, in spite of its worthlessness (it literally adds no power to the language, just sugar) and strangeness because of the perceived benefits to readability.
I have a bit of an unusual perspective; I ended up writing some drafts of the PEP, but not because I wanted decorators. Rather, I wanted to avoid a really bad syntax. (@ wasn't my favorite, but it is way better than several of the earlier proposals -- particularly in terms of what it doesn't assume about the future.) Almost any syntax is a big improvement *for*the*relevant*use*cases*. The concern is what it would do to python in general, and which future alternatives it would cut off.
And in my opinion, there are similar cases here, where readability is improved by having the definition of a function come after the expression to which the function will be passed.
Some of the examples looked like the order was an improvement. But are there huge areas where this is needed all over the place? (Extension frameworks such as Cocoa were mentioned for decorators.) Is it useful across several types of programming? (I see it as obvious for callbacks, but maybe not so clearcut elsewhere.) What is the cost of not doing it? With decorators, that cost was often delaying the wrapper for 20-30 lines, and then having to repeat the function name three times, and then using a code style that does confuse some people. In this case, I think the cost of not doing it is more like "Think up a name for the 2-3 line function, and then move the definition up ahead of the usage." Moving the definition is annoying, but not so horrible with small functions, and there is no need to repeat names. (I suppose you could argue that there is one repetition, if the function will never be used elsewhere.) What would the proposed syntax cost? In the case of @, that cost was reduced to burning a punctuation character, and explaining that functions can be annotated -- and the annotation is more than a comment. For the currently proposed syntax, it confuses several issues with suites (blocks). It sounds like the parser can probably handle it, but it isn't as clear that humans can. It might well preclude later block extensions, including more powerful versions, such as thunks or macros. (And it still burns a punctuation character anyhow.)
Yet, there's a lot of resistance to this, and I'm not entirely sure why. If it's just a problem with using "&" and "do" other line-noise and keywords can be proposed to substitute for it.
Yes. Or "using:", of course. The catch is that those are all so generic that this is only one of a zillion possible meanings -- and for most people, not the most obvious. A more specific keyword might have a better chance. (Not a *good* chance -- there is quite a high bar for keywords -- but still *better*.)
$ has no meaning in Python, for example.
I think it starts to with string templates. (And I think "!" is used by some popular extensions, such as scipy, and I think @ interfered with the leo editor, and ...) I'm not saying it is ambiguous to the parser, but the universe of unused ascii-punctuation isn't really all that huge.
If it's the lack of function names and docstrings, there's no reason these lambda-like-thingies can't have support for names and docstrings added (as in my old proposal, for example).
That wouldn't help unless people actually used the names and docstrings -- which again brings up the question about why existing use of functions as first class objects isn't enough.
But I feel like there's some deeper reason people don't think this is a readability win. What is it?
Cost-benefit. It is harder to see the costs, because they're spread out thinly all over the language, while the benefits are concentrated. But those costs are important. In this case, it looks like the benefits are smaller than for decorators, and the costs (at least with the proposed syntax) are higher. And decorators were not a shoe-in. -jJ
Carl Johnson writes:
The thing about this debate that confuses me is why was the @ decorator ever approved in the first place?
Decorators serve the same kind of purpose that define-syntax does in Scheme, and are similarly hygenic (which is a technical term in Scheme that more or less means "the rope is too short to get all the way around your neck"). They are *not* worthless to me; they allow boilerplate to be given a name, and to be applied to transform the behavior of a function at definition time rather than invocation time. By giving them a syntactic implementation, it becomes possible and encouraged to associate them with the declaration of a function, rather than placing them at some arbitrary point later in the file. (I can just see somebody inventing a style where all the decorators are defined and applied at the very end of the file, for example.) Agreed, that said, "@" doesn't evoke any association with the concept of decoration, but that's a problem of documentation. So I guess you had to be there to understand it....
And yet, @ was approved, in spite of its worthlessness (it literally adds no power to the language, just sugar)
But that is true of *all* operators. Lisp does just fine with no operators at all.
and strangeness because of the perceived benefits to readability. And in my opinion, there are similar cases here, where readability is improved by having the definition of a function come after the expression to which the function will be passed.
Right. That's your opinion, and as you've seen the majority of those posting to this thread disagree that the examples given are more readable. Even more distressing, all though he hasn't posted, GvR almost certainly would be included in them. On the other hand, most Pythonistas agree that @compute_some_boilerplate(variant) def foo (): # 20 lines of code is more readable than def foo (): # 20 lines of code foo = compute_some_boilerplate(foo, variant)
But I feel like there's some deeper reason people don't think this is a readability win. What is it?
Some people feel that giving names to a cognitive chunk is an essential part of the process of chunking. They are made uncomfortable, and often confused, by code which is named only by its own source (aka "blocks"). Others don't feel that way. The former oppose this kind of proposal strongly, the latter support it more or less actively. Guido apparently is in the former camp. But I think the deciding factor most likely is that it is the opinion of the BDFL that the harm done by allowing blocks to the cognition of the "please give it a name" group is much greater than the convenience to the "I don't need a name when I've got the code in front of me" group.
Paul Moore wrote:
(note that I could make my version look a bit nicer by a "from functools import partial", so you're still not allowed to argue aesthetics :-))?
Make that a "from functools import partial as do" and you get: @do(fn) def _(...): stmt1 stmt2 @do(fn, a, b, c) def _(...): stmt1 stmt2 Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Chris Perkins wrote:
On Tue, Jul 14, 2009 at 7:41 AM, Carl Johnson<cmjohnson.mailinglist@gmail.com> wrote:
This idea seems to be almost identical to my thread from November, "Proposal for Ruby-style anonymous block functions (that don't kill the indention)": http://mail.python.org/pipermail/python-ideas/2008-November/002340.html
Cool! I had not seen that thread, but the fact that you and I have proposed nearly the same thing independently increases the odds that I'm not just nuts. :)
The real difference is that your proposal uses & (which is a bad idea, since & already has a meaning: __and__) and in
foo() do: BODY
it implicitly injects the BODY function into the args of foo, which violates Explicit Is Better Than Implicit.
I considered the use of a "magic" placeholder character to be an unfortunate but necessary evil - the alternative seemed to be to restrict the block/def-expression to becoming the last argument to the function call that it follows, and that was just too restrictive. I have no particular attachment to "&" - but note that there is no ambiguity with the bitwise and operator, as far as I can tell. Eg. this works:
def f(a): return a() ... 3 & f(&) do: return 5 1
(not that you would write real code like that...)
[snip] Another possibility is to permit a local name (it's not an anonymous function any more!) so that you can have multiple functions: 3 & f(foo) def foo: return 5 3 & f(foo, bar) def foo, bar: return 5 and: return 6 Anyway, how would you handle condition expressions, as used in 'if' and 'while' statements? /They/ also expect a terminating colon and an indented block.
On Tue, Jul 14, 2009 at 11:26 AM, MRAB<python@mrabarnett.plus.com> wrote:
Anyway, how would you handle condition expressions, as used in 'if' and 'while' statements? /They/ also expect a terminating colon and an indented block.
Python 2.7a0 (trunk, Jul 10 2009, 16:43:32) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
def f(a): return a() ... if f() do: return True: File "<stdin>", line 1 if f() do: return True: ^ SyntaxError: invalid syntax while (f() do: return 1): File "<stdin>", line 1 while (f() do: return 1): ^ SyntaxError: invalid syntax
Yeah, so the error message could be clearer :) But I think I have covered all the bases. I think. That's why I'm hoping at least a few people will apply the patch and find the edge cases that I have inevitably missed. To answer your question more specifically: the modification to the grammar allows a block following a "simple_stmt" only. Here's the relevant bit: stmt: simple_stmt | compound_stmt # additional restrictions enforced by the interpreter - only certain # types of small_stmt are allowed to have a block attached: simple_stmt: small_stmt (';' small_stmt)* (([';'] NEWLINE) | block) block: 'do' [parameters] ':' suite The cases you are concerned about are "compount_stmt"s, so a block is not allowed. Chris Perkins
participants (24)
-
Aahz -
Ben Finney -
Carl Johnson -
Chris Perkins -
Chris Rebert -
Daniel Stutzbach -
David Stanek -
George Sakkis -
Gerald Britton -
Greg Ewing -
Guido van Rossum -
Jae Kwon -
Jan Kaliszewski -
Jim Jewett -
Mathias Panzenböck -
Mike Meyer -
MRAB -
Nick Coghlan -
Oleg Broytmann -
Paul Moore -
Scott David Daniels -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy