Re: [Python-ideas] [Python-Dev] Language proposal: variable assignment in functional context
Welcome Robert. My response below. Follow-ups to Python-Ideas, thanks. You'll need to subscribe to see any further discussion. On Fri, Jun 16, 2017 at 11:32:19AM +0000, Robert Vanden Eynde wrote:
In a nutshell, I would like to be able to write: y = (b+2 for b = a + 1)
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be: y = b + 2 given: b = a + 1 https://www.python.org/dev/peps/pep-3150/ In mathematics, I might write: y = b + 2 where b = a + 1 although of course I wouldn't do so for anything so simple. Here's a better example, the quadratic formula: -b ± √Δ x = ───────── 2a where Δ = b² - 4ac although even there I'd usually write Δ in place.
Python already have the "functional if", lambdas, list comprehension, but not simple assignment functional style.
I think you mean "if *expression*" rather than "functional if". The term "functional" in programming usually refers to a particular paradigm: https://en.wikipedia.org/wiki/Functional_programming -- Steve
On 17.06.2017 02:27, Steven D'Aprano wrote:
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1
Just to get this right:this proposal is about reversing the order of chaining expressions? Instead of: b = a + 1 c = b + 2 we could write it in reverse order: c = b + 2 given/for: b = a + 1 If so, I don't know if it just complicates the language with a feature which does not save writing nor reading nor cpu cycles nor memory and which adds a functionality which is already there (but in reverse order). Maybe there are more benefits I don't see right now. Sven
On Sat, Jun 17, 2017 at 09:03:54AM +0200, Sven R. Kunze wrote:
On 17.06.2017 02:27, Steven D'Aprano wrote:
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1
Just to get this right:this proposal is about reversing the order of chaining expressions?
Partly. Did you read the PEP? https://www.python.org/dev/peps/pep-3150/ I quote: The primary motivation is to enable a more declarative style of programming, where the operation to be performed is presented to the reader first, and the details of the necessary subcalculations are presented in the following indented suite. [...] A secondary motivation is to simplify interim calculations in module and class level code without polluting the resulting namespaces. It is not *just* about reversing the order, it is also about avoiding polluting the current namespace (global, or class) with unnecessary temporary variables. This puts the emphasis on the important part of the expression, not the temporary/implementation variables: page = header + body + footer where: header = ... body = ... footer = ... There is prior art: the "where" and "let" clauses in Haskell, as well as mathematics, where it is very common to defer the definition of temporary variables until after they are used.
Instead of:
b = a + 1 c = b + 2
we could write it in reverse order:
c = b + 2 given/for: b = a + 1
Right. But of course such a trivial example doesn't demonstrate any benefit. This might be a better example. Imagine you have this code, where the regular expression and the custom sort function are used in one place only. Because they're only used *once*, we don't really need them to be top-level global names, but currently we have little choice. regex = re.compile(r'.*?(\d*).*') def custom_sort(string): mo = regex.match(string) ... some implementation return key # Later results = sorted(some_strings, key=custom_sort) # Optional del custom_sort, regex Here we get the order of definitions backwards: the thing we actually care about, results = sorted(...), is defined last, and mere implementation details are given priority as top-level names that either hang around forever, or need to be explicitly deleted. Some sort of "where" clause could allow: results = sorted(some_strings, key=custom_sort) where: regex = re.compile(r'.*?(\d*).*') def custom_sort(string): mo = regex.match(string) ... some implementation return key If this syntax was introduced, editors would soon allow you to fold the "where" block and hide it. The custom_sort and regex names would be local to the where block and the results = ... line. Another important use-case is comprehensions, where we often have to repeat ourselves: [obj[0].field.method(spam)[eggs] for obj in sequence if obj[0].field.method] One work around: [m(spam)[eggs] for m in [obj[0].field.method for obj in sequence] if m] But perhaps we could do something like: [m(spam)[eggs] for obj in sequence where m = obj[0].field.method if m] or something similar.
If so, I don't know if it just complicates the language with a feature which does not save writing nor reading
It helps to save reading, by pushing less-important implementation details of an expression into an inner block where it is easy to ignore them. Even if you don't have an editor which does code folding, it is easy to skip over an indented block and just read the header line, ignoring the implementation. We already do this with classes, functions, even loops: class K: ... implementation of K def func(arg): ... implementation of func for x in seq: ... implementation of loop body page = header + body + footer where: ... implementation of page As a general rule, any two lines at the same level of indentation are read as being of equal importance. When we care about the implementation details, we "drop down" into the lower indentation block. But when skimming the code for a high-level overview, we skip the details of indented blocks and focus only on the current level: class K: def func(arg): for x in seq: page = header + body + footer where: (That's why editors often provide code folding, to hide the details of an indented block. But even without that feature, we can do it in our own head, although not as effectively.) -- Steve
On 17 June 2017 at 17:03, Sven R. Kunze <srkunze@mail.de> wrote:
If so, I don't know if it just complicates the language with a feature which does not save writing nor reading nor cpu cycles nor memory and which adds a functionality which is already there (but in reverse order).
Maybe there are more benefits I don't see right now.
You've pretty much hit on why that PEP's been deferred for ~5 years or so - I'm waiting to see use cases where we can genuinely say "this would be so much easier and more readable if we had a given construct!" :) One of the original motivations was that it may potentially make writing callback based code easier. Then asyncio (and variants like curio and trio) came along and asked the question: what if we built on the concepts explored by Twisted's inlineDeferred's, and instead made it easier to write asynchronous code without explicitly constructing callback chains? I do still think the idea has potential (and Steven's post does a good job of summarising why), since mathematical discussion includes the "statement (given these assumptions: form)" for a reason, and pragmatically such a clause offers an interim "single-use namespace" refactoring step between "inline mess of spaghetti code" and "out of order execution using a named function". However, in my own work, having to come up with a sensible name for the encapsulated operation generally comes with a readability benefit as well, so... Cheers, Nick. P.S. One potentially interesting area of application might be in SymPy, as it may make it possible to write symbolic mathematical expressions that track almost identically with their conventionally written counterparts. That's not as compelling a use case as PEP 465's matrix multiplication, but it's also not hard to be more compelling than the limited set of examples I had previously collected :) -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I might not have understood it completely, but I think the use cases would probably better be splitted into two categories, each with a simple solution (simple in usage at least): *When we just want a tiny scope for a variable:* Syntax: with [assignment]: # use of the variable # assigned variable is now out of scope Examples: with b = a + 1: y = b + 2 # we can use y here, but not b or with delta = lambda a, b, c: b**2 - 4 * a * c: x1 = (- b - math.sqrt(delta(a, b, c))) / (2 * a) x2 = (- b + math.sqrt(delta(a, b, c))) / (2 * a) # delta func is now out of scope and has been destroyed We don't keep unnecessarily some variables, as well as we don't risk any collision with outer scopes (and we preserve readability by not declaring a function for that). It would probably mean the assignment operator should behave differently than it does now which could have unexpected (to me) implications. It would have to support both __enter__ and __exit__ methods, but I don't know if this makes any sense. I don't know if with a + 1 as b: would make a better sense or be a side-effect or special cases hell. *When we want to simplify a comprehension:* (although it would probably help in many other situations) Syntax: prepare_iterable(sequence, *functions) which creates a new iterable containing tuples like (element, return_of_function_1, return_of_function_2, ...) Examples: [m(spam)[eggs] for _, m inprepare_iterable(sequence, lambda obj: obj[0].field.method) if m] or, outside of a comprehension: sequence = [0, 1, 5] prepare_iterable(sequence, lambda o: o * 3, lambda o: o + 1) # -> [(0, 0, 1), (1, 3, 2), (5, 15, 6)] The "prepare_iterable" method name might or might not be the right word to use. But English not being my mother language, I'm not the right person to discuss this... It would be a function instead of a method shared by all iterables to be able to yield the elements instead of processing the hole set of data right from the start. This function should probably belong to the standard library but probably not in the general namespace. -- Brice Le 17/06/17 à 12:27, Steven D'Aprano a écrit :
On Sat, Jun 17, 2017 at 09:03:54AM +0200, Sven R. Kunze wrote:
On 17.06.2017 02:27, Steven D'Aprano wrote:
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1 Just to get this right:this proposal is about reversing the order of chaining expressions? Partly. Did you read the PEP?
https://www.python.org/dev/peps/pep-3150/
I quote:
The primary motivation is to enable a more declarative style of programming, where the operation to be performed is presented to the reader first, and the details of the necessary subcalculations are presented in the following indented suite. [...] A secondary motivation is to simplify interim calculations in module and class level code without polluting the resulting namespaces.
It is not *just* about reversing the order, it is also about avoiding polluting the current namespace (global, or class) with unnecessary temporary variables. This puts the emphasis on the important part of the expression, not the temporary/implementation variables:
page = header + body + footer where: header = ... body = ... footer = ...
There is prior art: the "where" and "let" clauses in Haskell, as well as mathematics, where it is very common to defer the definition of temporary variables until after they are used.
Instead of:
b = a + 1 c = b + 2
we could write it in reverse order:
c = b + 2 given/for: b = a + 1
Right. But of course such a trivial example doesn't demonstrate any benefit. This might be a better example.
Imagine you have this code, where the regular expression and the custom sort function are used in one place only. Because they're only used *once*, we don't really need them to be top-level global names, but currently we have little choice.
regex = re.compile(r'.*?(\d*).*')
def custom_sort(string): mo = regex.match(string) ... some implementation return key
# Later results = sorted(some_strings, key=custom_sort)
# Optional del custom_sort, regex
Here we get the order of definitions backwards: the thing we actually care about, results = sorted(...), is defined last, and mere implementation details are given priority as top-level names that either hang around forever, or need to be explicitly deleted.
Some sort of "where" clause could allow:
results = sorted(some_strings, key=custom_sort) where: regex = re.compile(r'.*?(\d*).*')
def custom_sort(string): mo = regex.match(string) ... some implementation return key
If this syntax was introduced, editors would soon allow you to fold the "where" block and hide it. The custom_sort and regex names would be local to the where block and the results = ... line.
Another important use-case is comprehensions, where we often have to repeat ourselves:
[obj[0].field.method(spam)[eggs] for obj in sequence if obj[0].field.method]
One work around:
[m(spam)[eggs] for m in [obj[0].field.method for obj in sequence] if m]
But perhaps we could do something like:
[m(spam)[eggs] for obj in sequence where m = obj[0].field.method if m]
or something similar.
If so, I don't know if it just complicates the language with a feature which does not save writing nor reading It helps to save reading, by pushing less-important implementation details of an expression into an inner block where it is easy to ignore them. Even if you don't have an editor which does code folding, it is easy to skip over an indented block and just read the header line, ignoring the implementation. We already do this with classes, functions, even loops:
class K: ... implementation of K
def func(arg): ... implementation of func
for x in seq: ... implementation of loop body
page = header + body + footer where: ... implementation of page
As a general rule, any two lines at the same level of indentation are read as being of equal importance. When we care about the implementation details, we "drop down" into the lower indentation block. But when skimming the code for a high-level overview, we skip the details of indented blocks and focus only on the current level:
class K: def func(arg): for x in seq: page = header + body + footer where:
(That's why editors often provide code folding, to hide the details of an indented block. But even without that feature, we can do it in our own head, although not as effectively.)
I could emulate the "where" semantics as described by Steven using the class statement and eval : I guess it is useful if someone want to try refactoring a piece of "real world" code, and see if it really feels better - then we could try to push for the "where" syntax, which I kinda like: (This emulation requires the final expression to be either a string to be eval'ed, or a lambda function - but then some namespace retrieval and parameter matching would require a bit more code on the metaclass): class Where(type): def __new__(metacls, name, bases, namespace, *, expr=''): return eval(expr, namespace) def sqroot(n): class roots(expr="[(-b + r)/ (2 * a) for r in (+ delta **0.5, - delta ** 0.5) ]", metaclass=Where): a, b, c = n delta = b ** 2 - 4 * a * c return roots On 21 June 2017 at 10:31, Brice PARENT <contact@brice.xyz> wrote:
I might not have understood it completely, but I think the use cases would probably better be splitted into two categories, each with a simple solution (simple in usage at least):
*When we just want a tiny scope for a variable:*
Syntax:
with [assignment]: # use of the variable
# assigned variable is now out of scope
Examples:
with b = a + 1: y = b + 2
# we can use y here, but not b
or
with delta = lambda a, b, c: b**2 - 4 * a * c: x1 = (- b - math.sqrt(delta(a, b, c))) / (2 * a) x2 = (- b + math.sqrt(delta(a, b, c))) / (2 * a)
# delta func is now out of scope and has been destroyed
We don't keep unnecessarily some variables, as well as we don't risk any collision with outer scopes (and we preserve readability by not declaring a function for that).
It would probably mean the assignment operator should behave differently than it does now which could have unexpected (to me) implications. It would have to support both __enter__ and __exit__ methods, but I don't know if this makes any sense. I don't know if with a + 1 as b: would make a better sense or be a side-effect or special cases hell.
*When we want to simplify a comprehension:*
(although it would probably help in many other situations)
Syntax:
prepare_iterable(sequence, *functions)
which creates a new iterable containing tuples like (element, return_of_function_1, return_of_function_2, ...)
Examples:
[m(spam)[eggs] for _, m in prepare_iterable(sequence, lambda obj: obj[0].field.method) if m]
or, outside of a comprehension:
sequence = [0, 1, 5] prepare_iterable(sequence, lambda o: o * 3, lambda o: o + 1) # -> [(0, 0, 1), (1, 3, 2), (5, 15, 6)] The "prepare_iterable" method name might or might not be the right word to use. But English not being my mother language, I'm not the right person to discuss this... It would be a function instead of a method shared by all iterables to be able to yield the elements instead of processing the hole set of data right from the start. This function should probably belong to the standard library but probably not in the general namespace.
-- Brice
Le 17/06/17 à 12:27, Steven D'Aprano a écrit :
On Sat, Jun 17, 2017 at 09:03:54AM +0200, Sven R. Kunze wrote:
On 17.06.2017 02:27, Steven D'Aprano wrote:
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1
Just to get this right:this proposal is about reversing the order of chaining expressions?
Partly. Did you read the PEP? https://www.python.org/dev/peps/pep-3150/
I quote:
The primary motivation is to enable a more declarative style of programming, where the operation to be performed is presented to the reader first, and the details of the necessary subcalculations are presented in the following indented suite. [...] A secondary motivation is to simplify interim calculations in module and class level code without polluting the resulting namespaces.
It is not *just* about reversing the order, it is also about avoiding polluting the current namespace (global, or class) with unnecessary temporary variables. This puts the emphasis on the important part of the expression, not the temporary/implementation variables:
page = header + body + footer where: header = ... body = ... footer = ...
There is prior art: the "where" and "let" clauses in Haskell, as well as mathematics, where it is very common to defer the definition of temporary variables until after they are used.
Instead of:
b = a + 1 c = b + 2
we could write it in reverse order:
c = b + 2 given/for: b = a + 1
Right. But of course such a trivial example doesn't demonstrate any benefit. This might be a better example.
Imagine you have this code, where the regular expression and the custom sort function are used in one place only. Because they're only used *once*, we don't really need them to be top-level global names, but currently we have little choice.
regex = re.compile(r'.*?(\d*).*')
def custom_sort(string): mo = regex.match(string) ... some implementation return key
# Later results = sorted(some_strings, key=custom_sort)
# Optional del custom_sort, regex
Here we get the order of definitions backwards: the thing we actually care about, results = sorted(...), is defined last, and mere implementation details are given priority as top-level names that either hang around forever, or need to be explicitly deleted.
Some sort of "where" clause could allow:
results = sorted(some_strings, key=custom_sort) where: regex = re.compile(r'.*?(\d*).*')
def custom_sort(string): mo = regex.match(string) ... some implementation return key
If this syntax was introduced, editors would soon allow you to fold the "where" block and hide it. The custom_sort and regex names would be local to the where block and the results = ... line.
Another important use-case is comprehensions, where we often have to repeat ourselves:
[obj[0].field.method(spam)[eggs] for obj in sequence if obj[0].field.method]
One work around:
[m(spam)[eggs] for m in [obj[0].field.method for obj in sequence] if m]
But perhaps we could do something like:
[m(spam)[eggs] for obj in sequence where m = obj[0].field.method if m]
or something similar.
If so, I don't know if it just complicates the language with a feature which does not save writing nor reading
It helps to save reading, by pushing less-important implementation details of an expression into an inner block where it is easy to ignore them. Even if you don't have an editor which does code folding, it is easy to skip over an indented block and just read the header line, ignoring the implementation. We already do this with classes, functions, even loops:
class K: ... implementation of K
def func(arg): ... implementation of func
for x in seq: ... implementation of loop body
page = header + body + footer where: ... implementation of page
As a general rule, any two lines at the same level of indentation are read as being of equal importance. When we care about the implementation details, we "drop down" into the lower indentation block. But when skimming the code for a high-level overview, we skip the details of indented blocks and focus only on the current level:
class K: def func(arg): for x in seq: page = header + body + footer where:
(That's why editors often provide code folding, to hide the details of an indented block. But even without that feature, we can do it in our own head, although not as effectively.)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Brice PARENT <contact@brice.xyz> writes:
Examples:
with b = a + 1: y = b + 2
I don't think that could work, because the with "arguments" should be expressions, not statements. However, IIRC someone already suggested the alternative with a+1 as b: y = b + 2 but that clashes with the ordinary "context manager" syntax. It's a pity "exec" is now a plain function, instead of a keyword as it was in Py2, as that could allow exec: y = b + 2 with: # or even "in:" b = a + 1 ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele@metapensiero.it | -- Fortunato Depero, 1929.
On 17.06.2017 17:51, Nick Coghlan wrote:
You've pretty much hit on why that PEP's been deferred for ~5 years or so - I'm waiting to see use cases where we can genuinely say "this would be so much easier and more readable if we had a given construct!" :)
This PEP accepted, we would have 3 ways of doing 1 thing (imperative programming): 1) flat inline execution namespace 2) structured inline execution namespace 3) named code used in 1) oder 2) Is that simple enough for Python? Just one side thought: internally, we have a guideline which says: "please reduce the number of indentations" -> less else, less if, less while, etc. The goal is more compact, less complex code. Our code is already complex enough due to its sher amount. "given" would fall under the same rule here: keep it flat and simple; otherwise, give it a name.
Then asyncio (and variants like curio and trio) came along and asked the question: what if we built on the concepts explored by Twisted's inlineDeferred's, and instead made it easier to write asynchronous code without explicitly constructing callback chains?
Does it relate? I can imagine having both "given" and "async given".
However, in my own work, having to come up with a sensible name for the encapsulated operation generally comes with a readability benefit as well, so...
Well said. In the end (when it comes to professional code), you need to test those little things anyway. So addressing them is necessary. In interactive code, well, honestly, I don't care so much about spoiling namespaces and using variables names such as 'a' or 'bb' is quite common to try things out. @Steven Good post, thanks for explaining it. :) Might be too much for the simple Python I know and value but hey if it helps. Maybe it will enable a lot of cool stuff and we cannot imagine right now just because David Beazley cannot try it out in practice. Regards, Sven
I've implemented a PoC of `where` expression some time ago. https://github.com/thektulu/cpython/commit/9e669d63d292a639eb6ba2ecea3ed2c0c... just compile and have fun. 2017-06-17 2:27 GMT+02:00 Steven D'Aprano <steve@pearwood.info>:
Welcome Robert. My response below.
Follow-ups to Python-Ideas, thanks. You'll need to subscribe to see any further discussion.
On Fri, Jun 16, 2017 at 11:32:19AM +0000, Robert Vanden Eynde wrote:
In a nutshell, I would like to be able to write: y = (b+2 for b = a + 1)
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1
https://www.python.org/dev/peps/pep-3150/
In mathematics, I might write:
y = b + 2 where b = a + 1
although of course I wouldn't do so for anything so simple. Here's a better example, the quadratic formula:
-b ± √Δ x = ───────── 2a
where Δ = b² - 4ac
although even there I'd usually write Δ in place.
Python already have the "functional if", lambdas, list comprehension, but not simple assignment functional style.
I think you mean "if *expression*" rather than "functional if". The term "functional" in programming usually refers to a particular paradigm:
https://en.wikipedia.org/wiki/Functional_programming
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Cool idea, Michał. I hope there's at least somebody willing to try it out in practice. On 23.06.2017 02:21, Michał Żukowski wrote:
I've implemented a PoC of `where` expression some time ago.
https://github.com/thektulu/cpython/commit/9e669d63d292a639eb6ba2ecea3ed2c0c...
just compile and have fun.
2017-06-17 2:27 GMT+02:00 Steven D'Aprano <steve@pearwood.info <mailto:steve@pearwood.info>>:
Welcome Robert. My response below.
Follow-ups to Python-Ideas, thanks. You'll need to subscribe to see any further discussion.
On Fri, Jun 16, 2017 at 11:32:19AM +0000, Robert Vanden Eynde wrote:
> In a nutshell, I would like to be able to write: > y = (b+2 for b = a + 1)
I think this is somewhat similar to a suggestion of Nick Coghlan's. One possible syntax as a statement might be:
y = b + 2 given: b = a + 1
https://www.python.org/dev/peps/pep-3150/ <https://www.python.org/dev/peps/pep-3150/>
In mathematics, I might write:
y = b + 2 where b = a + 1
although of course I wouldn't do so for anything so simple. Here's a better example, the quadratic formula:
-b ± √Δ x = ───────── 2a
where Δ = b² - 4ac
although even there I'd usually write Δ in place.
> Python already have the "functional if", lambdas, list comprehension, > but not simple assignment functional style.
I think you mean "if *expression*" rather than "functional if". The term "functional" in programming usually refers to a particular paradigm:
https://en.wikipedia.org/wiki/Functional_programming <https://en.wikipedia.org/wiki/Functional_programming>
-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas <https://mail.python.org/mailman/listinfo/python-ideas> Code of Conduct: http://python.org/psf/codeofconduct/ <http://python.org/psf/codeofconduct/>
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
participants (7)
-
Brice PARENT -
Joao S. O. Bueno -
Lele Gaifax -
Michał Żukowski -
Nick Coghlan -
Steven D'Aprano -
Sven R. Kunze