Re: [Python-ideas] Where-statement (Proposal for function expressions)
Daniel Stutzbach wrote:
Unlikely, but this is:
total =3D sum(f(item) for item in lst) where: def f(item): return item.size if item.size > 0 else 1
Yes, that seems much more sensible than trying to repeat the where's inside of list comp or gen exp. Daniel Stutzbach wrote:
How about the following as additional syntactic sugar for the common one-= function case?
x =3D blah(f) where def f(item): body_of_f
I like it. Gerald Britton wrote:
Why not just use the Haskell approach?
foo(x,y) =3D myfunc(bar) where: =A0 =A0 =A0 =A0 =A0 myfunc, bar =3D f(x), g(y) where: =A0 =A0 =A0 =A0 =A0 =A0 =A0f,g =3D func1, func2
Because I've been looking at that for a couple minutes and still have no idea what it is supposed to mean. :-( List comprehensions and pattern matching are the only Haskell features that make any sense to me. Jan Kaliszewski wrote:
I don't like the direction of limiting content of the block to set of sin= gular simple statements. I suppose where-block should simply create a separ= ate nested scope.
Yes. It strikes me that one of the problems with Ruby is that there are different scoping rules for blocks, lambda, methods, procs, and whatever else=85 It would be much easier for users if "where" blocks had the exact same scoping rules as functions. Daniel Stutzbach's suggested equivalence strikes me as exactly correct: [return|yield|x=3D] expression_list where: suite is roughly equivalent to: def where_expression(): suite return expression_list [return|yield|x=3D] where_expression() -- Carl Johnson
foo(x,y) = myfunc(bar) where: myfunc = lambda x:f(x) bar = lambda y:g (y) where: f = lambda x:getattr(x, 'func') g = lambda y:str(y) would translate to: def foo(x,y): return getattr(x, 'func')str(y) Basically Haskell reads like math: circle_area = pi * radius**2 where pi is 3.14159 and radius is the radius of the circle You unpack it as you go. You could do the same this way: f = lambda x:getattr(x,'func') g = lambda y:str(y) bar = lambda y: g(y) myfunc = lambda x: f(x) foo(x,y) = lambda x,y: myfunc(x)(bar(y)) though this is just a toy example to give the general flavor. On Wed, Jul 15, 2009 at 7:12 PM, Carl Johnson<cmjohnson.mailinglist@gmail.com> wrote:
Daniel Stutzbach wrote:
Unlikely, but this is:
total =3D sum(f(item) for item in lst) where: def f(item): return item.size if item.size > 0 else 1
Yes, that seems much more sensible than trying to repeat the where's inside of list comp or gen exp.
Daniel Stutzbach wrote:
How about the following as additional syntactic sugar for the common one-= function case?
x =3D blah(f) where def f(item): body_of_f
I like it.
Gerald Britton wrote:
Why not just use the Haskell approach?
foo(x,y) =3D myfunc(bar) where: =A0 =A0 =A0 =A0 =A0 myfunc, bar =3D f(x), g(y) where: =A0 =A0 =A0 =A0 =A0 =A0 =A0f,g =3D func1, func2
Because I've been looking at that for a couple minutes and still have no idea what it is supposed to mean. :-( List comprehensions and pattern matching are the only Haskell features that make any sense to me.
Jan Kaliszewski wrote:
I don't like the direction of limiting content of the block to set of sin= gular simple statements. I suppose where-block should simply create a separ= ate nested scope.
Yes. It strikes me that one of the problems with Ruby is that there are different scoping rules for blocks, lambda, methods, procs, and whatever else=85 It would be much easier for users if "where" blocks had the exact same scoping rules as functions. Daniel Stutzbach's suggested equivalence strikes me as exactly correct:
[return|yield|x=3D] expression_list where: suite
is roughly equivalent to:
def where_expression(): suite return expression_list [return|yield|x=3D] where_expression()
-- Carl Johnson _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Gerald Britton
Gerald Britton <gerald.britton@gmail.com> (GB) wrote:
GB> foo(x,y) = myfunc(bar) where: GB> myfunc = lambda x:f(x) GB> bar = lambda y:g (y) GB> where: GB> f = lambda x:getattr(x, 'func') GB> g = lambda y:str(y)
GB> would translate to:
GB> def foo(x,y): GB> return getattr(x, 'func')str(y)
I don't think so. In the original code x and y after the where are only used as bound variables in the lambda's so they are not the x and y of the foo def. You could replace all of them by p and q without changing the meaning. If you replace all lambda x: fun(x) by the equivalent fun then you get this: def foo(x,y): return getattr(str, 'func') x and y are not used.
GB> Basically Haskell reads like math:
GB> circle_area = pi * radius**2 where pi is 3.14159 and radius is the GB> radius of the circle
GB> You unpack it as you go. You could do the same this way:
GB> f = lambda x:getattr(x,'func') GB> g = lambda y:str(y)
GB> bar = lambda y: g(y) GB> myfunc = lambda x: f(x)
GB> foo(x,y) = lambda x,y: myfunc(x)(bar(y))
Huh? How do you come at this step? IMHO myfunc(bar) is not the same as lambda x,y: myfunc(x)(bar(y)). It would be the same as (lambda x: myfunc(x))(lambda y: bar(y)) -- Piet van Oostrum <piet@cs.uu.nl> URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4] Private email: piet@vanoostrum.org
Huh? How do you come at this step? IMHO myfunc(bar) is not the same as lambda x,y: myfunc(x)(bar(y)). It would be the same as (lambda x: myfunc(x))(lambda y: bar(y))
Possibly, I wrote it up in a hurry. Perhaps this one is clearer: foo = lambda (x,y): myfunc(bar) where: myfunc = getattr(x, 'func') bar = str(y) Should be the same as: foo = lambda (x,y): getattr(x, 'func')(str(y)) The "where" clause is just syntactic sugar but can make some things easier to follow. Even in the toy example above it is tough to (visually) parse: getattr(x, 'func')(str(y)) and some feel that breaking it down bit by bit is easier. So the example could be written: def foo(x,y): myfunc = getattr(x, 'func') bar = str(y) return myfunc(bar) It really comes down to a matter of taste. Should the programmer be forced to define things in execution order (like the def foo, above) or should there be freedom to define things when you absolutely need to (the example with the "where" clause)? Another example: area(x) = pi * r ** 2 where: import math pi = math.pi r = circle.get_radius() where: circle = Circle(object) where: class Circle(radius): def __init__(self, radius): self.radius = radius def get_radius(self): return self.radius I know, it starts to look absurd after a while. However, I would use it if available to make long expressions more readable. For example, say I have a database class with many functions with descriptive names that are a bit long. Let's set it up this way: class MyDataBase(DB): def __init__(self, path): self.path = path ... def get_records_with_key(self, key): rec = db.get_by_key(key) while rec: yield rec rec = db.get_next_by_key(key) Now, in my code I might have a snippet: employees = MyDataBase('employees.db') .... smiths = [record for record in db.get_records_with_key('smith')] If I has a "where" clause, I'd prefer to write this: smiths = [record for record in stuff] where: stuff = db.get_records_with_key('smith') Of course there are other ways to write this in standard Python, but I don't think that's the point of this thread. I think that adding a "where" clause would make it possible to write deep list comprehensions and other long expressions without lots of parentheses. I suspect that I'm not alone in messing up either the placement or number of parentheses or both. Here's another one. Remember that a generator expression sometimes needs parentheses and sometimes not? For example: from itertools import chain c = chain(i for i in [2,3,5,7,11]) # no parentheses around the generator d = chain( (i for i in [2,3,5]) , (j for j in [7,11] ) # needs parentheses It's easy to mess up the extra parentheses (I sure do (and did, above!)) This might be nicer: d = chain(g1, g2) where: g1 = (i for i in [2,3,5]) g1 = (j for j in [7,11]) Even though there are just as many parentheses, they're broken out and easier to spot. Sure, you could define g1 and g2 ahead of time, but if the "where" clause creates a new namespace like a function def in a function, then g1 and g2 are just placeholders in the expression and don't pollute the variable namespace of the calling context. Anyway, I don't feel strongly either way. I see the benefits of "where" clauses and I'd use them if they were available, but I can certainly do without.
On Thu, Jul 16, 2009 at 9:39 AM, Gerald Britton<gerald.britton@gmail.com> wrote:
d = chain(g1, g2) where: g1 = (i for i in [2,3,5]) g1 = (j for j in [7,11])
Even though there are just as many parentheses, they're broken out and easier to spot. Sure, you could define g1 and g2 ahead of time, but if the "where" clause creates a new namespace like a function def in a function, then g1 and g2 are just placeholders in the expression and don't pollute the variable namespace of the calling context.
This idea is really growing on me. Here's another example - how do you calculate sample variance? You're probably thinking "Um, well, I think it's the square root of the sum of the squares minus the square of the sums". Great, that's a high-ish level, intuitive description of the algorithm. The precise details of how to go about calculating that are not important yet. So write it that way: def variance(data): return sqrt(sum_of_squares - square_of_sums) where: ... elided ... There - we've written the core of the algorithm at the same level of abstraction as we think about it. The code matches our mental model. Only then do we go about filling in the boring details. def variance(data): return sqrt(sum_of_squares - square_of_sums) where: def sqrt(v): return v ** 0.5 sum_of_squares = sum(v**2 for v in data) square_of_sums = sum(data) ** 2 The advantage is that now readers of the code, having developed some intuition about what "where blocks" mean, can see at a glance that the code at the first level of indentation is the important stuff, and the indented block is the implementation details; the stuff that's at a lower level of abstraction. I think this construct would be a great way to enable SLAP - the Same Level of Abstraction Principle (which I'm embarrasingly in love with). Traditional SLAP says that all the code in a function should be at the same level of abstraction - code at a lower level of abstraction should be split out into other functions or methods. A refinement could could state that code at a lower level of abstraction should be either moved to another function, or indented in a where block.
Anyway, I don't feel strongly either way. I see the benefits of "where" clauses and I'd use them if they were available, but I can certainly do without.
I would use them too, and I can't do without. I'm going to hold my breath until we get them. ;) As for the precise specification, I see only a few issues. As a first approximation, assume that this code: [LHS OP] EXPR where: BODY is expanded into the equivalent of this: def TEMP(): BODY return EXPR [LHS OP] TEMP() That seems like the most obvious way to get a nested scope to run the body in. Problem 1) return Since there will be an implicit return injected into the body of TEMP, what happens if the user puts an explicit return in the body of the block: x = foo(y) where: return 23 expands to def TEMP(): return 23 return foo(y) x = TEMP() So return should be disallowed. Problem 2) break/continue for a in range(b): x = foo(y) where: break #or continue expands to: for a in range(b): def TEMP(): break # or continue return foo(y) x = TEMP() Which is an error. However: x = foo(y) where: y = [] for i in range(n): if cond(i): break y.append(i) must be allowed (right?). So we can't make a blanket restriction on break/continue within the body. Problem 3) yield x = foo(y) where: yield 3 expands to: def TEMP(): yield 3 return foo(y) x = TEMP() which is an error. So yield must be disallowed. I can't think of any other constructs that would need special handling, and I'm sure the issues I have listed are solvable. Chris Perkins
2009/7/16 Chris Perkins <chrisperkins99@gmail.com>:
As for the precise specification, I see only a few issues.
As a first approximation, assume that this code:
[LHS OP] EXPR where: BODY
is expanded into the equivalent of this:
def TEMP(): BODY return EXPR [LHS OP] TEMP()
So where constructs are only allowed for code of the form [LHS OP] EXPR? What does that correspond to in the grammar? (If it doesn't correspond to something at the moment, you'll need to extend the grammar to define a "thing that can have a where clause" production...) You've also disallowed x[i] = 12 where: i = some_complicated_expression() Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-) Paul.
On Thu, Jul 16, 2009 at 11:06 AM, Paul Moore<p.f.moore@gmail.com> wrote:
So where constructs are only allowed for code of the form [LHS OP] EXPR?
What does that correspond to in the grammar? (If it doesn't correspond to something at the moment, you'll need to extend the grammar to define a "thing that can have a where clause" production...)
Without having thought too hard about it, I would allow a where block after a "simple_stmt" - same thing I did with my "do block" proposal. That means you could have a where after any of these: small_stmt: (expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | exec_stmt | assert_stmt) So by EXPR above, I guess I was thinking of any "small_stmt" production. Of course, it makes no sense for some of them, like pass_stmt, global_stmt, etc, but I don't see any harm in allowing the block to be there - I don't think I could have any effect though.
You've also disallowed
x[i] = 12 where: i = some_complicated_expression()
Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-)
Well, my original answer was "no, I just didnt' think of that'. But now that you mention it, I think in the case of an "augassign" production, it makes sense to have the left hand side evaluated in the current scope (not in the 'where block" scope). So in your examle, you would either get a NameError for i, the code would do somthing that you didn't expect. Chris Perkins
2009/7/16 Chris Perkins <chrisperkins99@gmail.com>:
You've also disallowed
x[i] = 12 where: i = some_complicated_expression()
Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-)
Well, my original answer was "no, I just didnt' think of that'. But now that you mention it, I think in the case of an "augassign" production, it makes sense to have the left hand side evaluated in the current scope (not in the 'where block" scope). So in your examle, you would either get a NameError for i, the code would do somthing that you didn't expect.
And yet the user's intent is obviously to use the i defined inside the where (that's how functional languages work, so it's hardly an unrealistic intent). I'd argue that having this example use a previously defined value of i, when x = i where: i = some_complicated_expression() uses the i defined in the where clause, is a disaster waiting to happen. On another point that came up - i = 5 print i # 5 x[i] = 12 where: global i i = 3 print i # 3 seems to me to be so unexpected and non-obvious that it makes the whole construct suspect. Yes, I know that the semantics, as given in terms of a rewrite, define what happens, but it seems to me that it goes counter to intuition. Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope. Currently, scopes are created by the class and def statements. Both of these define objects (classes and functions, respectively). The where clause as you've stated it defines a scope, but no object. That doesn't make it ill-defined, but it does make it (very, in my view) non-intuitive. Paul.
16-07-2009, 20:51 Paul Moore <p.f.moore@gmail.com> wrote:
x[i] = 12 where: i = some_complicated_expression() [snip] And yet the user's intent is obviously to use the i defined inside the where (that's how functional languages work, so it's hardly an unrealistic intent). [snip] Yes, I know that the semantics, as given in terms of a rewrite, define what happens, but it seems to me that it goes counter to intuition.
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope. Currently, scopes are created by the class and def statements. Both of these define objects (classes and functions, respectively). The where clause as you've stated it defines a scope, but no object. That doesn't make it ill-defined, but it does make it (very, in my view) non-intuitive.
Yeah, It is an issue :-/ I can imagine some solutions based on name overriding rules, e.g.: * in the "target line" only those names are treated as part of nested where-block scope, which are targets of assignment within the where-block (as MRAB proposes, if I understood well), or * in the "target line" only those names are treated as part of nested where-block scope, which are not targets of assignment within the target line. But it seems to me obscure at first sight, and I don't know if implementation wouldn't be problematic... Maybe the solution is to make 'where' creating an object, somehow similarly to the way the 'class' statement works? Some examples: x[somename.i] = 12 where somename: i = some_complicated_expression() def variance(data): # for some definition of "variance" return var.sqrt(var.sum_of_squares(data) - var.square_of_sums(data)) where var: def sqrt... def sum_of_squares... def square_of_sums... foo(numbers.x, numbers.y) where numbers: x = a * 42 y = b / 17 a = {} a[fn.key] = bar(fn.f, fn.g) where fn: key = ... def f(x): ... def g(y): ... Another possibility is to allow using "anonymus name": x[.index] = 12 where: index = some_complicated_expression() foo(.x, .y) where: x = a * 42 y = b / 17 Cheers, zuo
On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope.
I think that's why the "where" clause is being proposed: because there's no construct to create a new scope without defining a new class or function. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
2009/7/16 Daniel Stutzbach <daniel@stutzbachenterprises.com>:
On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope.
I think that's why the "where" clause is being proposed: because there's no construct to create a new scope without defining a new class or function.
That's a very interesting perspective. Maybe then it would make more sense to simply bite the bullet and define a scope-creating statement - scope: i = a_complicated_expression() x[i] = 15 Without a way of having some values "escape" the scope, this may be a bit limited, though. And it's almost certainly not going to appeal to the people who want to see the expression first, and the subordinate names defined afterwards. But that's a different underlying concept - a new construct to execute code in something other than top-to-bottom order (and one I have less interest in, personally). Paul.
Paul Moore wrote:
2009/7/16 Daniel Stutzbach <daniel@stutzbachenterprises.com>:
On Thu, Jul 16, 2009 at 1:51 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope. I think that's why the "where" clause is being proposed: because there's no construct to create a new scope without defining a new class or function.
That's a very interesting perspective. Maybe then it would make more sense to simply bite the bullet and define a scope-creating statement -
scope: i = a_complicated_expression() x[i] = 15
Without a way of having some values "escape" the scope, this may be a bit limited, though. And it's almost certainly not going to appeal to the people who want to see the expression first, and the subordinate names defined afterwards. But that's a different underlying concept - a new construct to execute code in something other than top-to-bottom order (and one I have less interest in, personally).
Perhaps: scope except foo: ... :-)
Daniel Stutzbach wrote:
I think that's why the "where" clause is being proposed: because there's no construct to create a new scope without defining a new class or function.
No, the reason is to provide a way to put implementation details in a convenient place after where they're used, but not too far away. -- Greg
On Thu, Jul 16, 2009 at 2:51 PM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/16 Chris Perkins <chrisperkins99@gmail.com>:
You've also disallowed
x[i] = 12 where: i = some_complicated_expression()
And yet the user's intent is obviously to use the i defined inside the where (that's how functional languages work, so it's hardly an unrealistic intent).
I don't follow this argument - in what way is that how functional languages work? I see that code as analogous to the following Clojure, for example: (def x (into-array [1 2])) (def i 0) (aset x i (let [i 1] 99)) You wouldn't expect x to now be [1 99], rather than [99 2], would you?
I'd argue that having this example use a previously defined value of i, when
x = i where: i = some_complicated_expression()
uses the i defined in the where clause, is a disaster waiting to happen.
I don't really see the problem with it. If you told people that the where block only applies to the right-hand-side of an assignment, and that the left-hand-side is always evaluated outside the scope of the where block, I doubt there would be much confusion.
On another point that came up -
i = 5 print i # 5 x[i] = 12 where: global i i = 3 print i # 3
seems to me to be so unexpected and non-obvious that it makes the whole construct suspect. Yes, I know that the semantics, as given in terms of a rewrite, define what happens, but it seems to me that it goes counter to intuition.
Again, I don't think that's a big deal. One solution is to disallow global statements (if we're already disallowing stuff like break and return, why not). Another is just "don't do that".
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope. Currently, scopes are created by the class and def statements. Both of these define objects (classes and functions, respectively). The where clause as you've stated it defines a scope, but no object. That doesn't make it ill-defined, but it does make it (very, in my view) non-intuitive.
"Namespaces are one honking great idea -- let's do more of those!" And everyone knows, you can't argue with the Zen. ;) Chris Perkins
2009/7/17 Chris Perkins <chrisperkins99@gmail.com>:
On Thu, Jul 16, 2009 at 2:51 PM, Paul Moore<p.f.moore@gmail.com> wrote:
2009/7/16 Chris Perkins <chrisperkins99@gmail.com>:
You've also disallowed
x[i] = 12 where: i = some_complicated_expression()
And yet the user's intent is obviously to use the i defined inside the where (that's how functional languages work, so it's hardly an unrealistic intent).
I don't follow this argument - in what way is that how functional languages work?
Sorry, I should have said "Haskell, and some other functional languages I vaguely recall working similarly" :-) In Haskell, the expressions given names in the where clause are used throughout the expression to the left of the where. Of course immutability and referential transparency makes that a lot less controversial...
I don't really see the problem with it. If you told people that the where block only applies to the right-hand-side of an assignment, and that the left-hand-side is always evaluated outside the scope of the where block, I doubt there would be much confusion.
That somewhat assumes that a where block is only used for assignments (or more accurately, that there's a special exception for the left side of an assignment not being included). That's at best a wart.
Again, I don't think that's a big deal. One solution is to disallow global statements (if we're already disallowing stuff like break and return, why not). Another is just "don't do that".
More warts. At some point, the weight of the various exceptions make the basic idea not worth it.
"Namespaces are one honking great idea -- let's do more of those!"
And everyone knows, you can't argue with the Zen. ;)
"If the implementation is hard to explain, it's a bad idea." "Simple is better than complex." But the Zen can have arguments with itself :-) Paul.
On Fri, Jul 17, 2009 at 9:23 AM, Paul Moore<p.f.moore@gmail.com> wrote:
I don't really see the problem with it. If you told people that the where block only applies to the right-hand-side of an assignment, and that the left-hand-side is always evaluated outside the scope of the where block, I doubt there would be much confusion.
That somewhat assumes that a where block is only used for assignments (or more accurately, that there's a special exception for the left side of an assignment not being included). That's at best a wart.
The "special case for assignments" is what I was assuming. In fact, I liked this proposal partly because the semantics seemed so obvious that you could guess them and be right - but looking through this thread, I see that the semantics are obvious to different people in different ways. For example, I see that many people seem to expect: i = 1 x[i] = 3 where: i = 2 to mean "x[2] = 3", which I would find utterly surprising.
Again, I don't think that's a big deal. One solution is to disallow global statements (if we're already disallowing stuff like break and return, why not). Another is just "don't do that".
More warts. At some point, the weight of the various exceptions make the basic idea not worth it.
That does worry me, I'll admit - if the code that you can legally write in a where block is only a subset of Python, then, well... that's just ugly. Nevertheless, I find several of the examples to have such a readability advantage - similar to the way that list comprehensions are immediately and wonderfully more readable than building a list in a for loop - that I'm still in favor of the idea. Chris Perkins
Paul Moore wrote:
That somewhat assumes that a where block is only used for assignments (or more accurately, that there's a special exception for the left side of an assignment not being included). That's at best a wart.
The more I think about it, the more I think it would be best for the where block *not* to start a new scope. That would completely eliminate all these scope issues, along with break/return/yield problems, etc. -- Greg
Paul Moore wrote:
Thinking about it, the issue is that you're making where the *only* example in Python (that i can think of) of a non-defining construct which creates a new scope.
Not quite true any more, since LCs and generator expressions now use a local scope for their iteration variables. However in this case, it might just be less confusing overall to not have a new scope for the where-block. It's not necessary in order to get the main intended benefit. -- Greg
On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore@gmail.com> wrote:
x[i] = 12 where: i = some_complicated_expression()
Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-)
Because only the right-hand side is in the where-block's scope. If the left-hand side were in the where-block's scope, then: x = 0 x = i where: i = 5 print x Would print "0" not "5". -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
16-07-2009, 17:37 Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore@gmail.com> wrote:
x[i] = 12 where: i = some_complicated_expression()
Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-)
Because only the right-hand side is in the where-block's scope.
Yeah. But I believe it should be possible using global/nonlocal keywords, e.g.: x[i] = 12 where: global i i = some_complicated_expression() I think that many rules that apply for functions should also apply for where-blocks. (But obviously not all of them, e.g. return, yield and such stuff don't make sense) 16-07-2009, 18:16 Chris Perkins <chrisperkins99@gmail.com> wrote:
Yes, I'm very interested to see what proportion of people find the top-down style more readable.
I think even for the same person it can strongly depend on particular application. It'll be good for some things, and useless for other. But it isn't a strong argument against the feature. IMHO it'll be useful in many situations. 15-07-2009, 14:49 Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
How about the following as additional syntactic sugar for the common one-function case? x = blah(f) where def f(item): body_of_f
+1 from me also. Cheers, -- Jan Kaliszewski <zuo@chopin.edu.pl>
On Thu, Jul 16, 2009 at 12:51 PM, Jan Kaliszewski<zuo@chopin.edu.pl> wrote:
How about the following as additional syntactic sugar for the common one-function case? x = blah(f) where def f(item): body_of_f
+1 from me also.
I'm not sold on that one yet, only because someone is inevitably going to ask why they can't create a singleton-ish sort of thing like this: thing = Thing() where class Thing(object): def __init__(...): etc Or maybe that should be allowed too? Chris Perkins
Jan Kaliszewski wrote:
16-07-2009, 17:37 Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
On Thu, Jul 16, 2009 at 10:06 AM, Paul Moore <p.f.moore@gmail.com> wrote:
x[i] = 12 where: i = some_complicated_expression()
Was that deliberate? If so, could you explain why now, so that it's on record for the legions of people who will ask after it's implemented? :-)
Because only the right-hand side is in the where-block's scope.
Yeah. But I believe it should be possible using global/nonlocal keywords, e.g.:
x[i] = 12 where: global i i = some_complicated_expression()
I think that many rules that apply for functions should also apply for where-blocks. (But obviously not all of them, e.g. return, yield and such stuff don't make sense)
16-07-2009, 18:16 Chris Perkins <chrisperkins99@gmail.com> wrote:
Yes, I'm very interested to see what proportion of people find the top-down style more readable.
I think even for the same person it can strongly depend on particular application. It'll be good for some things, and useless for other. But it isn't a strong argument against the feature. IMHO it'll be useful in many situations.
15-07-2009, 14:49 Daniel Stutzbach <daniel@stutzbachenterprises.com> wrote:
How about the following as additional syntactic sugar for the common one-function case? x = blah(f) where def f(item): body_of_f
+1 from me also.
[Apologies for the word 'assignment' in what follows. :-)] When defining a function, any name that's the target of an assignment will default to being local; other names will be non-local. In a 'where' construct the rule could be that any name that's the target of an assignment before the 'where' will be non-local; other names will default to local if they are the target of an assignment within the 'where' construct, but non-local otherwise. For example: bar = "outside" foo = bar where: bar = "inside" 'foo' is non-local. There's an assignment to 'bar' with the 'where' construct, so 'bar' is local in the statement and hides the non-local 'bar'. Therefore 'foo' becomes bound to "inside". More clearly: bar = "outside" local: nonlocal foo bar = "inside" foo = bar Another example: x[i] = 12 where: i = some_complicated_expression() 'x' is non-local because there's no assignment to 'x' within the 'where' construct'. 'i' is local because there is an assignment to 'i' within the 'where' construct'. More clearly: local: i = some_complicated_expression() x[i] = 12
On Thu, Jul 16, 2009 at 1:20 PM, MRAB <python@mrabarnett.plus.com> wrote:
bar = "outside" local: nonlocal foo bar = "inside" foo = bar
"nonlocal" requires that "foo" is already defined in an enclosing scope. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
Daniel Stutzbach wrote:
If the left-hand side were in the where-block's scope, then:
x = 0 x = i where: i = 5 print x
Would print "0" not "5".
My intuitive expectations are different for assignment to a bare name vs. assignment to an item or attribute. In x = i where: i = 5 I would expect x to be bound in the current scope, not the where-block scope. But I would expect x[i] = 5 where: x = y i = 7 to be equivalent to y[7] = 5 i.e. both x and i are *evaluated* in the where-block scope. I'm not sure where this leaves us with respect to augmented assignment, though. If you follow it to its logical conclusion, then x += i where: i = 5 should be equivalent to x = x.__iadd__(i) where: i = 5 and then the evaluation and rebinding of 'x' would happen in different scopes! -- Greg
On Fri, 17 Jul 2009 14:02:34 +1200 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If the left-hand side were in the where-block's scope, then:
x = 0 x = i where: i = 5 print x
Would print "0" not "5".
My intuitive expectations are different for assignment to a bare name vs. assignment to an item or attribute. In
x = i where: i = 5
I would expect x to be bound in the current scope, not the where-block scope. But I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
i.e. both x and i are *evaluated* in the where-block scope.
I'm not sure where this leaves us with respect to augmented assignment, though. If you follow it to its logical conclusion, then
x += i where: i = 5
should be equivalent to
x = x.__iadd__(i) where: i = 5
and then the evaluation and rebinding of 'x' would happen in different scopes!
That evaluating an expression to the right of an augmented assignment might return something different than what you would bind that expression to has always been an issue with augmented assignments. I believe this particular problem belongs has more to do with them than with the proposed where statement. I think the intuitive behavior doesn't involve position relative to the assignment, but binding vs. non-bindings. What you *expect* is that expressions left of the where will be evaluated in the inner scope (created by the where), but that any bindings that happen there will happen in the outer scope (before the where). That makes what you want to happen happen - *most* of the time: i = 3 x += i where: i = 5 increments x by 5. i = 'a' x[i] = 'b' where: i = 'c' sets x['c'] to 'b'. Things get really ugly when the bound variables start appearing in both scopes combined with augmented assignments: x = 3 x += 4 where: x = 5 print x What should this print? <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
17-07-2009, 04:25 Mike Meyer <mwm-keyword-python.b4bdba@mired.org> wrote:
On Fri, 17 Jul 2009 14:02:34 +1200 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If the left-hand side were in the where-block's scope, then:
x = 0 x = i where: i = 5 print x
Would print "0" not "5".
My intuitive expectations are different for assignment to a bare name vs. assignment to an item or attribute. In
x = i where: i = 5
I would expect x to be bound in the current scope, not the where-block scope. But I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
i.e. both x and i are *evaluated* in the where-block scope.
I'm not sure where this leaves us with respect to augmented assignment, though. If you follow it to its logical conclusion, then
x += i where: i = 5
should be equivalent to
x = x.__iadd__(i) where: i = 5
and then the evaluation and rebinding of 'x' would happen in different scopes! [snip] I think the intuitive behavior doesn't involve position relative to the assignment, but binding vs. non-bindings. What you *expect* is that expressions left of the where will be evaluated in the inner scope (created by the where), but that any bindings that happen there will happen in the outer scope (before the where).
That makes what you want to happen happen - *most* of the time:
i = 3 x += i where: i = 5
increments x by 5.
i = 'a' x[i] = 'b' where: i = 'c'
sets x['c'] to 'b'.
Things get really ugly when the bound variables start appearing in both scopes combined with augmented assignments:
x = 3 x += 4 where: x = 5 print x
What should this print?
I think, a good rule could be that: All *expressions* of the target line are evaluated within the where-block scope; and -- moreover -- those names (variables) that are *bound* in the target line, are "leaked" to the outer scope. [target line == the line with where statement] Then the former example code should print 9. -- Jan Kaliszewski <zuo@chopin.edu.pl>
Mike Meyer wrote:
On Fri, 17 Jul 2009 14:02:34 +1200 Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Daniel Stutzbach wrote:
If the left-hand side were in the where-block's scope, then:
x = 0 x = i where: i = 5 print x
Would print "0" not "5". My intuitive expectations are different for assignment to a bare name vs. assignment to an item or attribute. In
x = i where: i = 5
I would expect x to be bound in the current scope, not the where-block scope. But I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
i.e. both x and i are *evaluated* in the where-block scope.
I'm not sure where this leaves us with respect to augmented assignment, though. If you follow it to its logical conclusion, then
x += i where: i = 5
should be equivalent to
x = x.__iadd__(i) where: i = 5
and then the evaluation and rebinding of 'x' would happen in different scopes!
That evaluating an expression to the right of an augmented assignment might return something different than what you would bind that expression to has always been an issue with augmented assignments. I believe this particular problem belongs has more to do with them than with the proposed where statement.
I think the intuitive behavior doesn't involve position relative to the assignment, but binding vs. non-bindings. What you *expect* is that expressions left of the where will be evaluated in the inner scope (created by the where), but that any bindings that happen there will happen in the outer scope (before the where).
That makes what you want to happen happen - *most* of the time:
i = 3 x += i where: i = 5
increments x by 5.
i = 'a' x[i] = 'b' where: i = 'c'
sets x['c'] to 'b'.
Things get really ugly when the bound variables start appearing in both scopes combined with augmented assignments:
x = 3 x += 4 where: x = 5 print x
What should this print?
The target line assigns to x, therefore x is not local. It should print 9. x = 3 local: nonlocal x x = 5 x += 4 # The target line.
On Thu, Jul 16, 2009 at 9:02 PM, Greg Ewing <greg.ewing@canterbury.ac.nz>wrote:
I would expect x to be bound in the current scope, not the where-block scope. But I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
How about if attempts to re-bind variables in outer scopes should throw an exception? Much like they already do in this example:
x = 5 def foo(): ... blah = x ... x = 6 ... foo() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in foo UnboundLocalError: local variable 'x' referenced before assignment
(rebinding is allowed if "global" or "nonlocal" is explicitly used, of course) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>
On Fri, Jul 17, 2009 at 4:20 AM, Daniel Stutzbach<daniel@stutzbachenterprises.com> wrote:
On Thu, Jul 16, 2009 at 9:02 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
I would expect x to be bound in the current scope, not the where-block scope. But I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
How about if attempts to re-bind variables in outer scopes should throw an exception? Much like they already do in this example:
x = 5 def foo(): ... blah = x ... x = 6 ... foo() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<stdin>", line 2, in foo UnboundLocalError: local variable 'x' referenced before assignment
(rebinding is allowed if "global" or "nonlocal" is explicitly used, of course)
This thread is a little confusing to me. I am failing to see how the where block makes anything clearer. The fact that a new scope is created and names are available during a singe line of execution scares me. I also worry that this will encourage code duplication. Any substantial logic should be rolling into a function not into a where block. -- David blog: http://www.traceback.org twitter: http://twitter.com/dstanek
David Stanek <dstanek@dstanek.com> writes:
This thread is a little confusing to me. I am failing to see how the where block makes anything clearer.
+1. I haven't found that any of the examples improve the clarity of the code over using a named function. -- \ “If I melt dry ice, can I swim without getting wet?” —Steven | `\ Wright | _o__) | Ben Finney
Ben Finney wrote:
David Stanek <dstanek@dstanek.com> writes:
This thread is a little confusing to me. I am failing to see how the where block makes anything clearer.
+1. I haven't found that any of the examples improve the clarity of the code over using a named function.
I presume you mean -1 on the proposal, as am I (strongly).
Terry Reedy <tjreedy@udel.edu> writes:
Ben Finney wrote:
David Stanek <dstanek@dstanek.com> writes:
This thread is a little confusing to me. I am failing to see how the where block makes anything clearer.
+1. I haven't found that any of the examples improve the clarity of the code over using a named function.
I presume you mean -1 on the proposal, as am I (strongly).
Yes, my support was for David's dissent :-) -- \ “If you see an animal and you can't tell if it's a skunk or a | `\ cat, here's a good saying to help: ‘Black and white, stinks all | _o__) right. Tabby-colored, likes a fella.’” —Jack Handey | Ben Finney
Ben Finney wrote:
I haven't found that any of the examples improve the clarity of the code over using a named function.
This isn't about unnamed functions. The functions defined in a where block still have names. -- Greg
Greg Ewing <greg.ewing@canterbury.ac.nz> writes:
Ben Finney wrote:
I haven't found that any of the examples improve the clarity of the code over using a named function.
This isn't about unnamed functions. The functions defined in a where block still have names.
So this is about introducing a new syntax for named functions? That drives it even further from clarity, in my estimation. -- \ “An idea isn't responsible for the people who believe in it.” | `\ —Donald Robert Perry Marquis | _o__) | Ben Finney
Ben Finney wrote:
So this is about introducing a new syntax for named functions?
No, the named functions are declared in exactly the same way as they ever were. It isn't even particularly about functions at all, that just happens to be one of its potential uses. It's about being able to put higher-level code ahead of lower-level code, without having to relegate the lower-level code to somewhere far away. -- Greg
Greg Ewing writes:
It's about being able to put higher-level code ahead of lower-level code, without having to relegate the lower-level code to somewhere far away.
In other words, we need better programmers' editors which can temporarily bring the low-level code close to the high-level code, not more syntax.
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
In other words, we need better programmers' editors which can temporarily bring the low-level code close to the high-level code, not more syntax.
That assumes that we only look at program code using such editors, and ignores all the time spent looking at the code on paper, on web sites, in messages to each other, etc. If the source code is hard enough to read that we need special tools to make it readable, there's not much point any more in pretending that it's plain text. I hope we never approach that. -- \ “I like my dental hygenist, I think she's very pretty; so when | `\ I go to have my teeth cleaned, while I'm in the waiting room I | _o__) eat an entire box of cookies.” —Steven Wright | Ben Finney
Ben Finney <ben+python@benfinney.id.au> wrote:
That assumes that we only look at program code using such editors, and ignores all the time spent looking at the code on paper, on web sites, in messages to each other, etc.
Is there a MIME type for Python code? Something a Web browser or email client could use to catch Python snippets, and interpose a reasonable edit/inspect mode for that code? Still wouldn't help with paper, but I can imagine other uses for such a registration. "text/python" seems reasonable, but the registration procedure seems to suggest "text/prs.python" ("part of products that are not distributed commercially"). Bill
Bill Janssen <janssen@parc.com> writes:
Is there a MIME type for Python code? Something a Web browser or email client could use to catch Python snippets, and interpose a reasonable edit/inspect mode for that code? Still wouldn't help with paper, but I can imagine other uses for such a registration.
My system-installed ‘/etc/mime.types’ has an entry for each of ‘application/x-python-code’ (for compiled bytecode) and ‘text/x-python’ (for Python source). -- \ “When we call others dogmatic, what we really object to is | `\ their holding dogmas that are different from our own.” —Charles | _o__) Issawi | Ben Finney
On Sat, 18 Jul 2009 03:50:11 pm Ben Finney wrote:
If the source code is hard enough to read that we need special tools to make it readable, there's not much point any more in pretending that it's plain text. I hope we never approach that.
But source code isn't plain text. It has syntax, and grammar. Source code is a structured collection of tokens -- you can't put arbitrary plain text in source code and expect it to execute, or compile. There's value in treating source code like (say) HTML or XML -- it's structured text, human readable, and in a pinch if all else fails human writable as well, but normally you're expected to use a smart editor that understands the tokens and structure of the file. (Except for those who think there's something virtuous about keeping track of nested tags yourself.) To a certain extent we already do that: people say "you'll be far more productive if you use vi or emacs than if you use Microsoft Notepad". We accept that you simply can't be a productive, professional programmer using Notepad. (It is a bonus that, if you're stuck without your proper tools, you can edit code with Notepad, or ed. Great. It would be nice to be able to repair a car with nothing but a screwdriver, but nobody honestly thinks that's practical.) And we do expect people to edit source code using a *text* editor, a program that understands about text elements like lines and characters: nobody edits source with a hex editor, or by direct manipulation of the bytes, so there's already precedence for increasing specialisation. Perhaps it's time for programmers to start using token editors instead of text editors. Of course it will be a big paradigm shift, and we'll need to think about how token files will interact with utilities like grep that expect text files, but none of these things are problems, they're opportunities. -- Steven D'Aprano
Ben Finney writes:
"Stephen J. Turnbull" <stephen@xemacs.org> writes:
In other words, we need better programmers' editors which can temporarily bring the low-level code close to the high-level code, not more syntax.
That assumes that we only look at program code using such editors, and ignores all the time spent looking at the code on paper, on web sites, in messages to each other, etc.
No, actually it is based on a belief that the problem of serializing a program in a readable way is in general intractible.
If the source code is hard enough to read that we need special tools to make it readable,
Source code is *not* plain text. In all modern languages it is explicitly tree-structured. For that reason, having special tools to help read it (tags, programmers' editors, IDEs) is a long-established practice. The point of the where clause is to allow that tree structure to be expressed in top-down fashion, keeping subordinate nodes "textually close" to the root. Problem is, *that only works for very flat semantics*, in fact only for a root node and one generation of descendents (and not very many of them, with none very complex, at that). In general, it's just not possible to keep nodes of the "where" tree close to *both* their parents *and* their siblings at high levels of the tree. It seems to me that even for people who like and use a device like the where clause will quickly run into its limitations. It is for *that* reason that I suggested that tools, not syntax, are the way to go here.
Stephen J. Turnbull <stephen@xemacs.org> wrote:
Problem is, *that only works for very flat semantics*, in fact only for a root node and one generation of descendents (and not very many of them, with none very complex, at that).
Agree that where clause is good for short and rather flat structures. The same thing is with list-comprehensions/generator expressions -- probably it is not a good idea to use them like that: ( 2*item for item in (seq4 for seq4 in sorted(seq3 for seq3 in set(seq2 for seq2 in filter(None, reversed(seq1 for seq1 in sorted(seq0 for seq0 in map(lambda x, y: y(x), adict0.keys(), adict0.values() ) if set(seq0) > set(adict0.values())) if len(seq1) > 3)) ) if seq3.issubset(adict3.items())) if seq4) ) :) Me wrote:
IMHO 'where' would be better than lambda in many contexts, especially if there is a need for function as argument, e.g. for filter(), map(), sort(key=...) etc., and also for some list comprehensions and generator expressions.
Of course, 'where' is not a statement/multi-line lambda, but in practice it'd be take role of it, being IMHO really useful. Still I'd prefer: a = sorted(b, key=how) where def how(item): # or even 'using' DO SOMETHING # instead of 'where def' return key ...rather than: def how(item): DO SOMETHING return key a = sorted(b, key=how) Mainly, not because I need it visually but because in such situations (say 'lambda-situations') I'm not interested *how to do* until I think about *what to do*. Regards, -- Jan Kaliszewski <zuo@chopin.edu.pl>
On 17 Jul 2009, at 03:02, Greg Ewing wrote:
[...] I would expect
x[i] = 5 where: x = y i = 7
to be equivalent to
y[7] = 5
i.e. both x and i are *evaluated* in the where-block scope.
So would I, given that the statement x[i] = 5 is the really the same as the expression x.__setitem__(i, 5)
I'm not sure where this leaves us with respect to augmented assignment, though. If you follow it to its logical conclusion, then
x += i where: i = 5
should be equivalent to
x = x.__iadd__(i) where: i = 5
and then the evaluation and rebinding of 'x' would happen in different scopes!
Indeed. But why not simply evaluate the whole statement in the where scope, with all names not bound in the where scope declared as nonlocal? i.e x[i], y = foo(a), b where: i = 7 def foo(x): return bar(x, x) would be equivalent to: def _(): nonlocal x, y, b i = 7 def foo(x): return bar(x, x) x[i], y = foo(a), b _() -- Arnaud
On Thu, Jul 16, 2009 at 10:51 AM, Chris Perkins<chrisperkins99@gmail.com> wrote:
This idea is really growing on me.
Here's another example - how do you calculate sample variance? You're probably thinking "Um, well, I think it's the square root of the sum of the squares minus the square of the sums". Great, that's a high-ish level, intuitive description of the algorithm. The precise details of how to go about calculating that are not important yet. So write it that way:
def variance(data): return sqrt(sum_of_squares - square_of_sums) where: ... elided ...
There - we've written the core of the algorithm at the same level of abstraction as we think about it. The code matches our mental model. Only then do we go about filling in the boring details.
def variance(data): return sqrt(sum_of_squares - square_of_sums) where: def sqrt(v): return v ** 0.5 sum_of_squares = sum(v**2 for v in data) square_of_sums = sum(data) ** 2
The advantage is that now readers of the code, having developed some intuition about what "where blocks" mean, can see at a glance that the code at the first level of indentation is the important stuff, and the indented block is the implementation details; the stuff that's at a lower level of abstraction.
I realize the appeal of this but I see two issues with it: - What you call an advantage can be a disadvantage for many programmers used to the standard bottom-up way most procedural and object-oriented languages support. For better or for worse, thinking sequentially feels more natural than in terms of top-down levels of abstractions. Unless one is (or gets) used to this style, "where" blocks would be the code equivalent of top-posting; it doesn't make sense unless you read it backwards. - TIMTOWTDI. In any case, bottom-up won't go away so for any non-trivial piece of code one has to choose between two equivalent ways of expressing it. Think of Perl's "STATEMENT (if|unless) (EXPR)". I'm not saying that these are necessarily show stoppers but they should be addressed to see whether the benefits are worth the added complexity. George
On Thu, Jul 16, 2009 at 12:05 PM, George Sakkis<george.sakkis@gmail.com> wrote:
- What you call an advantage can be a disadvantage for many programmers used to the standard bottom-up way most procedural and object-oriented languages support. For better or for worse, thinking sequentially feels more natural than in terms of top-down levels of abstractions. Unless one is (or gets) used to this style, "where" blocks would be the code equivalent of top-posting; it doesn't make sense unless you read it backwards.
That's interesting - I see it as exactly the opposite. For me, bottom-up approach appears backwards, and I get annoyed by having to read backwards. It clearly depends on what you're used to, I guess.
- TIMTOWTDI. In any case, bottom-up won't go away so for any non-trivial piece of code one has to choose between two equivalent ways of expressing it. Think of Perl's "STATEMENT (if|unless) (EXPR)".
I'm not saying that these are necessarily show stoppers but they should be addressed to see whether the benefits are worth the added complexity.
Yes, I'm very interested to see what proportion of people find the top-down style more readable. Chris Perkins
On Fri, 17 Jul 2009 12:51:54 am Chris Perkins wrote:
Here's another example - how do you calculate sample variance? You're probably thinking "Um, well, I think it's the square root of the sum of the squares minus the square of the sums". Great, that's a high-ish level, intuitive description of the algorithm. The precise details of how to go about calculating that are not important yet. So write it that way:
def variance(data): return sqrt(sum_of_squares - square_of_sums) where: ... elided ...
That would actually be the standard deviation, not the variance.
There - we've written the core of the algorithm at the same level of abstraction as we think about it. The code matches our mental model. Only then do we go about filling in the boring details.
def variance(data): return sqrt(sum_of_squares - square_of_sums) where: def sqrt(v): return v ** 0.5 sum_of_squares = sum(v**2 for v in data) square_of_sums = sum(data) ** 2
I don't know what you've calculated, but it's not the variance, or even the standard deviation. The variance is actually the mean of the squares of the deviations from the mean of the data. Written another way, that's the mean of the squares of the data minus the square of the mean. But putting all that aside, let's pretend that the formula you gave is correct. It's certainly calculating *something* -- let's pretend it's the variance. I'm now going to try to convince you that this entire approach is actually harmful and should be avoided. You said: "...sum of the squares minus the square of the sums". That's all very well, but what squares? What sums? You have under-specified what the function should do. Is it squares of the first fifty prime numbers? Sums of odd-positioned data? You can't just pretend that this is an implementation detail -- you have to face up to what the function actually does at the design phase. If you had, you would have noted that the so-called "variance" is the sum of the squares *of the data*, minus the square of the sums *of the data*. The sums-and-squares are functions of data, not expressions or constants, which suggests writing them as functions. This gives us: def variance(data): # for some definition of "variance" return sqrt(sum_of_squares(data) - square_of_sums(data)) where: ... This naturally suggests that they are (or could be) reusable functions that belong in the module namespace, not repeated inside each function that uses them: def sqrt(v): return v**0.5 def sum_of_squares(data): return sum(v**2 for v in data) def square_of_sums(data): return sum(data)**2 These can now easily be documented and tested, which is a major win. Having done this, the "where" statement is redundant: def variance(data): return sqrt(sum_of_squares(data) - square_of_sums(data)) Not just redundant, but actively harmful: it encourages the coder to duplicate common code inside functions instead of factoring it out into a single external function. This not only violates Don't Repeat Yourself, but it makes it harder to test and document the code. Now of course it's possible to inappropriately inline common code inside functions without "where". For instance, I might have written: def variance(data): def sqrt(v): return v**0.5 def sum_of_squares(data): return sum(v**2 for v in data) def square_of_sums(data): return sum(data)**2 return sqrt(sum_of_squares(data) - square_of_sums(data)) which is a micro-pessimation (not only do I lose the opportunity to re-use and test the internal functions, but I also pay the micro-cost of re-creating them every time I call the function). Or: def variance(data): sum_of_squares = sum(v**2 for v in data) square_of_sums = sum(data)**2 return (sum_of_squares - square_of_sums)**0.5 In either case, writing the sums-and-squares before the return suggests that these are common code best written as re-usable functions. But "where" changes the focus of the coder away from factoring code *out* of functions into placing code *inside* internal blocks. It encourages the coder to think of sum_of_squares as a private implementation detail instead of a re-usable component. And that is, I believe, harmful. -- Steven D'Aprano
16-07-2009, 19:13 Steven D'Aprano <steve@pearwood.info> wrote:
Not just redundant, but actively harmful: it encourages the coder to duplicate common code inside functions instead of factoring it out into a single external function.
But note that, if it's necessary, refactoring such code moving appropriate parts e.g. into global scope could be done very easily.
Now of course it's possible to inappropriately inline common code inside functions without "where".
Every language feature can be misused. More powerful feature -- larger possibilities of misusing. But language is not for substituting programmer's sense. Cheers, -- Jan Kaliszewski <zuo@chopin.edu.pl>
On Thu, Jul 16, 2009 at 1:13 PM, Steven D'Aprano<steve@pearwood.info> wrote:
On Fri, 17 Jul 2009 12:51:54 am Chris Perkins wrote:
def variance(data): return sqrt(sum_of_squares - square_of_sums) where: ... elided ...
That would actually be the standard deviation, not the variance.
Hence my use of "Um, I think it's..." :) Yeah, I couldn't remember if this was right, but as you say below, it's not the point of the example, so I didn't bother to look it up.
I'm now going to try to convince you that this entire approach is actually harmful and should be avoided. You said:
[ .. snip me being almost - but not quite - convinced ...]
def variance(data): def sqrt(v): return v**0.5 def sum_of_squares(data): return sum(v**2 for v in data) def square_of_sums(data): return sum(data)**2 return sqrt(sum_of_squares(data) - square_of_sums(data))
This is the case that I was trying to present an alternative to. The fact is, people _do_ write code like this. Sometimes pulling the local stuff out into reusable functions is the right thing to do, but I don't think it always is.
In either case, writing the sums-and-squares before the return suggests that these are common code best written as re-usable functions. But "where" changes the focus of the coder away from factoring code *out* of functions into placing code *inside* internal blocks. It encourages the coder to think of sum_of_squares as a private implementation detail instead of a re-usable component. And that is, I believe, harmful.
But surely you agree that _sometimes_ there is code that really is an implementation detail - not everything is reusable, or at least not usefully reusable. Taking your argument to the extreme, every function would be only one line of code. I prefer to trust programmers to make the best use of the tools available to create good code. In other words: Jan Kaliszewski:
Every language feature can be misused. More powerful feature -- larger possibilities of misusing. But language is not for substituting programmer's sense.
Yeah, ditto to what he said. Chris Perkins
On Fri, 17 Jul 2009 03:58:21 am Chris Perkins wrote:
In either case, writing the sums-and-squares before the return suggests that these are common code best written as re-usable functions. But "where" changes the focus of the coder away from factoring code *out* of functions into placing code *inside* internal blocks. It encourages the coder to think of sum_of_squares as a private implementation detail instead of a re-usable component. And that is, I believe, harmful.
But surely you agree that _sometimes_ there is code that really is an implementation detail - not everything is reusable, or at least not usefully reusable.
Of course. What your suggestion gives us, though, is a second way of including multi-line functions: def parrot(): x = 1 y = 2 return x + y versus def parrot(): return x + y where: x = 1 y = 2 I'm not convinced that allowing the second is either necessary or beneficial.
Taking your argument to the extreme, every function would be only one line of code. I prefer to trust programmers to make the best use of the tools available to create good code.
The Python community takes the attitude that we're all consenting adults here, and that if you want to shoot yourself in the foot, you should be allowed to. And that's fine. But it's only fine because Python, by and large, is a safe language, one with very few jagged edges, which encourages good practices rather than bad. Which brings me to this:
In other words:
Jan Kaliszewski:
Every language feature can be misused. More powerful feature -- larger possibilities of misusing. But language is not for substituting programmer's sense.
Yeah, ditto to what he said.
Some language features are begging to be misused, and although I've occasionally wanted a way to define code blocks on the fly outside of a function, I'm leaning towards the conclusion that this suggestion is a feature where the abuses outweigh the benefits. -- Steven D'Aprano
Chris Perkins wrote:
Which is an error. However: x = foo(y) where: y = [] for i in range(n): if cond(i): break y.append(i) must be allowed (right?). So we can't make a blanket restriction on break/continue within the body.
Same with return and yield if there's a def inside the where. (Seems obvious, but you never mentioned it.) -- Carl Johnson
[ Maybe a little late, but nobody else seems to have noticed this...] On Thu, 16 Jul 2009 10:51:54 -0400 Chris Perkins <chrisperkins99@gmail.com> wrote:
As a first approximation, assume that this code:
This seems reasonable, but there are a couple of questions. Key words: "first approximation".
[LHS OP] EXPR where: BODY
is expanded into the equivalent of this:
def TEMP(): BODY return EXPR [LHS OP] TEMP()
Problem 1) return Since there will be an implicit return injected into the body of TEMP, what happens if the user puts an explicit return in the body of the block: x = foo(y) where: return 23 expands to def TEMP(): return 23 return foo(y) x = TEMP() So return should be disallowed. . [ ...] Problem 3) yield x = foo(y) where: yield 3 expands to: def TEMP(): yield 3 return foo(y) x = TEMP() which is an error. So yield must be disallowed.
I disagree with both these conclusions. Clearly, they are paired, as they serve a similar purpose, but I don't see that they need to be disallowed at all. That they cause problems with your first attempt at describing an implementation certainly isn't reason to disallow them all by itself. On the other hand, using a where block to create some temporary names doesn't in any way prevent me from discovering in the middle of calculating their values that I have a value to return (or yield) from the enclosing function. This would pretty clearly be a wart should it be so. That said - if where can't be implemented in such a way as to allow a return or yield from inside a where, I don't believe that's sufficient reason to reject where out of hand. But we shouldn't start by assuming we have to break it. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba@mired.org> wrote:
That they cause problems with your first attempt at describing an implementation certainly isn't reason to disallow them all by itself.
But, it isn't a problem of implementation. I don't see any reasonable *semantics* of return or yield within where-block in proposed form. return/yield statements seem to be completely unnecessary in this context. (Of course, I don't say about return/yield statements within functions defined within where-block). -- Jan Kaliszewski <zuo@chopin.edu.pl>
Jan Kaliszewski wrote:
17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba@mired.org> wrote:
That they cause problems with your first attempt at describing an implementation certainly isn't reason to disallow them all by itself.
But, it isn't a problem of implementation. I don't see any reasonable *semantics* of return or yield within where-block in proposed form. return/yield statements seem to be completely unnecessary in this context. (Of course, I don't say about return/yield statements within functions defined within where-block).
I think that a return, yield, break or continue in a where-block would in a sense be no different to one in a normal suite, except, of course, that there happens to be a local namespace. foo = bar where: if condition: return bar = "bar" would be equivalent to: local: nonlocal foo if condition: return bar = "bar" foo = bar (I accept that 'nonlocal' might not be the right keyword, but you get the idea.)
On Thu, Jul 16, 2009 at 8:04 PM, Jan Kaliszewski<zuo@chopin.edu.pl> wrote:
... I don't see any reasonable *semantics* of return or yield within where-block in proposed form.
next_batch = myfilter(candidates, failed) where: failed=[] for x in this_batch: if magical(x): # We actually found one!!! return x # more failures, of course. # bias the next batch to a different section # of the search space. failed.append(x) I leave to others the judgment on whether or not any function containing code is already too long... alternatively, provide(foo(item)) where: start_to_generate_an_item() if flag(): # Are there reasons to return instead of throwing? raise HigherPriority item=finish_generation() Again, I'm not certain that the style should be encouraged, but neither I am sure it shouldn't be. -jJ
On Fri, 17 Jul 2009 02:04:05 +0200 "Jan Kaliszewski" <zuo@chopin.edu.pl> wrote:
17-07-2009, 00:51 Mike Meyer <mwm-keyword-python.b4bdba@mired.org> wrote:
That they cause problems with your first attempt at describing an implementation certainly isn't reason to disallow them all by itself.
But, it isn't a problem of implementation. I don't see any reasonable *semantics* of return or yield within where-block in proposed form. return/yield statements seem to be completely unnecessary in this context. (Of course, I don't say about return/yield statements within functions defined within where-block).
Seems obvious to me: def foo(x): bar = foundbar where: if x is None: return -1 foundbar = x * 2 return bar The return will return -1 from foo if it's invoked; the assignment to bar (and foundbar, for that matter) will never happen. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information. O< ascii ribbon campaign - stop html mail - www.asciiribbon.org
Mike Meyer wrote:
I disagree with both these conclusions. Clearly, they are paired, as they serve a similar purpose, but I don't see that they need to be disallowed at all. That they cause problems with your first attempt at describing an implementation certainly isn't reason to disallow them all by itself.
Let me toss out an alternate implementation. statement where: body creates a new temporary local context and evaluates the body in it. Any variables assigned to within the body are set in this temporary context (as you'd expect). Variables in the outer context are still visible to the body. Now the difference: Then the statement is run within this temporary local context, but with a change in that now all variables assigned to bypass the temporary context (perhaps unbinding the variable there, if it's set, so that it no longer hides the outer value?) and are set in the outer context. But variables within the temporary context are still visible to the statement. This is a new use of contexts for Python. Once the statement is finished executing, the temporary context is discarded. Thus: i = 2 x[27] = 2 x[i] = 4 where: i = 27 print(x[27]) # 4 print(i) # 2 has access to the temporary i (set to 27). While: x = 1 x = i where: i = 27 print(x) # 27 assigns 27 to x in the outer context. Thus, the where clause applies to the whole statement, without limiting the statement's ability to bind variables in the outer context. -Bruce
participants (20)
-
Arnaud Delobelle -
Ben Finney -
Bill Janssen -
Bruce Frederiksen -
Carl Johnson -
Chris Perkins -
Daniel Stutzbach -
David Stanek -
George Sakkis -
Gerald Britton -
Greg Ewing -
Jan Kaliszewski -
Jim Jewett -
Mike Meyer -
MRAB -
Paul Moore -
Piet van Oostrum -
Stephen J. Turnbull -
Steven D'Aprano -
Terry Reedy