Accessing the result of comprehension's expression from the conditional
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Even if f() is not a function such as
[x + 1 for x in l if x + 1 > 0]
it looks ugly since we're repeating ourself.
We can work around it like this:
[y for y in (f(x) for x in l) if y > 0]
but then we have an unnecessary nested loop comprehension.
I'm suggesting about something like:
[f(x) for x in l if @ > 0]
where @ is the result of the listcomp's expression (i.e. f(x))
Personally, I don't like the use of symbols like @, as python is not perl. I'm still thinking of a better syntax/approach and is open for suggestion.
What do you guys think?
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
Aahz wrote:
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
I think of it not as limitation but as an odd gap in functionality. I think having the semantics that the filtering is done after the expression part would be much more useful than the current behavior (filtering before expression).
If filtering is done after expression, we can access both the original and the transformed objects (it may also be possible to optimize the cases where the filter does not use the transformed objects, although this complex behavior wouldn't be pythonic)
Try rewriting this:
res = [x**x as F for x in nums if F < 100] (note: this is my new preferred syntax)
Attempts:
- Using a regular for-loop
res = [] for x in nums: F = x**x if F < 100: res.append(F)
remarks: five lines that's much more difficult to understand than a single, concise expression in standard form.
- Using nested comprehension
res = [F for F in (x**x for x in nums) if F < 100]
remarks: using nested list comprehension is DRY (on a loose definition of DRY principle). I have to repeat the comprehension body twice, when I only need one expression and one filtering.
- Using map()
res = [F for F in map(lambda x: x**x, nums) if F < 100]
remarks: when the expression part is just a simple expression, you have to use lambda and that's plain ugly. Not to mention when you want to filter based on both x and F.
Advantages of the proposal: - shorter - faster, as looping is done in C - more readable. The main advantage of comprehension is that it have standardized form, which is easier to understand, unlike a for-loop which can have an infinite number of variations. - (unnecessary) nested comprehension is an abuse. - with `as` keyword, no new keyword and no ambiguity since currently `as` cannot exist inside comprehension.
Disadvantages: - reverses the current semantic of filtering-then-expression. This shouldn't be too much problem since side-effect on the expression part is a cardinal sin and... - if the expression part is heavy, it might be possible to do optimization by filtering first when the filter part does not require the result (i.e. when there is no "as" clause). A good side effect of this optimization is codes that relies on filtering being done before expression will just work as they cannot contain an `as` keyword. (As "simple is better than complex", I actually don't really like `as` can change evaluation order; I much prefer to keep everything simple and consistent, i.e. always evaluate expression then filter or otherwise)
possible syntaxes: - [x**x as F for x in nums if F < 100] the as keyword is already often used to rename things (in with, import, etc) I like this one much better than @. The as part, of course, is optional - [x**x for x in nums if @ < 100] the initial proposed syntax, ugly as hell.
On Fri, Jun 19, 2009 at 12:39 AM, Lie Ryanlie.1296@gmail.com wrote:
Aahz wrote:
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
I think of it not as limitation but as an odd gap in functionality. I think having the semantics that the filtering is done after the expression part would be much more useful than the current behavior (filtering before expression).
If filtering is done after expression, we can access both the original and the transformed objects (it may also be possible to optimize the cases where the filter does not use the transformed objects, although this complex behavior wouldn't be pythonic)
Try rewriting this:
res = [x**x as F for x in nums if F < 100] (note: this is my new preferred syntax)
Attempts:
- Using a regular for-loop
res = [] for x in nums: F = x**x if F < 100: res.append(F)
remarks: five lines that's much more difficult to understand than a single, concise expression in standard form.
Depends on how complicated the "standard form" is; right now, it (list comps) is/are relatively simple. Perl syntax is concise and adheres to a standard, but is not easy to understand. Not to equate a minor syntax addition to Perl, but every addition is a step in that general direction. The road to hell was paved with good intentions.
Cheers, Chris -- http://blog.rebertia.com
2009/6/19 Lie Ryan lie.1296@gmail.com:
res = [x**x as F for x in nums if F < 100] (note: this is my new preferred syntax)
[...]
Advantages of the proposal:
- shorter
- faster, as looping is done in C
- more readable. The main advantage of comprehension is that it have
standardized form, which is easier to understand, unlike a for-loop which can have an infinite number of variations.
- (unnecessary) nested comprehension is an abuse.
- with `as` keyword, no new keyword and no ambiguity since currently
`as` cannot exist inside comprehension.
Disadvantages:
- reverses the current semantic of filtering-then-expression. This
shouldn't be too much problem since side-effect on the expression part is a cardinal sin and...
- if the expression part is heavy, it might be possible to do
optimization by filtering first when the filter part does not require the result (i.e. when there is no "as" clause). A good side effect of this optimization is codes that relies on filtering being done before expression will just work as they cannot contain an `as` keyword. (As "simple is better than complex", I actually don't really like `as` can change evaluation order; I much prefer to keep everything simple and consistent, i.e. always evaluate expression then filter or otherwise)
possible syntaxes:
- [x**x as F for x in nums if F < 100]
the as keyword is already often used to rename things (in with, import, etc) I like this one much better than @. The as part, of course, is optional
- [x**x for x in nums if @ < 100]
the initial proposed syntax, ugly as hell.
OK, with this explanation (and the new syntax) I see what you're getting at better.
However, changing the order of evaluate vs filter is a huge compatibility problem. There's no way this will be possible. Even with syntax triggering the change (so that it's one way with the "as", the other without), that's a disaster waiting to happen.
You have at least 3 explicit ways of stating your intent (genexp inside listcomp, map inside listcomp, explicit loop). None is as clean-looking as your (amended) proposal, but they work now, and they don't have the semantic issues of your proposal.
(For a more general, more radical, equally certain to be shot down, option, which at least doesn't introduce the change in semantics, you could try proposing "as" as an assignment-as-expression operator. So you could have
[y for x in l if (f(x) as y) < 100]
Hmm, on second thoughts - no, don't bother... :-))
Paul.
Paul Moore wrote:
2009/6/19 Lie Ryan lie.1296@gmail.com:
res = [x**x as F for x in nums if F < 100] (note: this is my new preferred syntax)
[...]
Advantages of the proposal:
- shorter
- faster, as looping is done in C
- more readable. The main advantage of comprehension is that it have
standardized form, which is easier to understand, unlike a for-loop which can have an infinite number of variations.
- (unnecessary) nested comprehension is an abuse.
- with `as` keyword, no new keyword and no ambiguity since currently
`as` cannot exist inside comprehension.
Disadvantages:
- reverses the current semantic of filtering-then-expression. This
shouldn't be too much problem since side-effect on the expression part is a cardinal sin and...
- if the expression part is heavy, it might be possible to do
optimization by filtering first when the filter part does not require the result (i.e. when there is no "as" clause). A good side effect of this optimization is codes that relies on filtering being done before expression will just work as they cannot contain an `as` keyword. (As "simple is better than complex", I actually don't really like `as` can change evaluation order; I much prefer to keep everything simple and consistent, i.e. always evaluate expression then filter or otherwise)
possible syntaxes:
- [x**x as F for x in nums if F < 100]
the as keyword is already often used to rename things (in with, import, etc) I like this one much better than @. The as part, of course, is optional
- [x**x for x in nums if @ < 100]
the initial proposed syntax, ugly as hell.
OK, with this explanation (and the new syntax) I see what you're getting at better.
However, changing the order of evaluate vs filter is a huge compatibility problem. There's no way this will be possible. Even with syntax triggering the change (so that it's one way with the "as", the other without), that's a disaster waiting to happen.
How about this syntax which would solve your concern for the semantic change:
[x**x as F for x in lst if F() < 100]
it's similar to original `as` proposal, except that F is a callable instead of direct value.
The advantage of F being callable is that it does not need semantic change, the filtering part will be done before expression just like it is right now. However, we can explicitly request for the expression to be evaluated by calling F(); the return value of F() will be saved and reused for the final result and other calls to F().
A diagrammatic explanation:
+--------------------------- | this is the part that name | the expression's callable | --+- [x**x as F for x in lst if x and F() < 100 and isvalid(F())] -+-- ---+---------------------------- | | -+- -+- | the filtering part is | | | | evaluated before | | | | expression just like | | | | current behavior | | | | -----------------------+ | | | | | | then when F gets called; | | | expression is evaluated, | | | cached, and returned | | | ---------------------------+ | | | | F is called again, return cached result | | ----------------------------------------+ | | at the end of the day, if F is called, | return the cached result, else evaluate | the expression and use that +-----------------------------------------
using the as-callable syntax, the semantic of this:
[f(x) as F for x in lst if g(F())]
would be similar to:
result = [] for x in lst: # F() ensures f(x) will be only ever be called once def F(): nonlocal _cache if not _cache: _cache = f(x) return _cache
_cache = None if g(F()): result.append(F())
the only disadvantage of this as-callable is if you forgot to call F.
You have at least 3 explicit ways of stating your intent (genexp inside listcomp, map inside listcomp, explicit loop). None is as clean-looking as your (amended) proposal, but they work now, and they don't have the semantic issues of your proposal.
(For a more general, more radical, equally certain to be shot down, option, which at least doesn't introduce the change in semantics, you could try proposing "as" as an assignment-as-expression operator. So you could have
[y for x in l if (f(x) as y) < 100]Hmm, on second thoughts - no, don't bother... :-))
It took me several minutes to understand that one... and no, that syntax as makes it way too easy to be too creative in list comprehension, devaluing the "standard form" which IMO is the strongest point of list comprehension. Not to mention that that syntax moved the expression part to be inside the filtering part... which is quite... disturbing...
On Sat, 20 Jun 2009 09:45:58 am Lie Ryan wrote:
How about this syntax which would solve your concern for the semantic change:
[x**x as F for x in lst if F() < 100]
Let's look at what that would be equivalent to.
L = [] for x in lst: F = lambda x=x: x**x # Need to use default value in the # lambda otherwise all elements will have the same value. tmp = F() if tmp < 100: L.append(tmp)
It's not clear why you think this is an improvement over:
[x**x as F for x in lst if F < 100] # note the missing ()s
which would be equivalent to:
L = [] for x in lst: F = x**x if F < 100: L.append(F)
Despite what you say here:
The advantage of F being callable is that it does not need semantic change, the filtering part will be done before expression just like it is right now.
That's not true. It can't be true. If you want to filter on x**x being greater than 100, you need to calculate x**x first. In theory, a sufficiently clever compiler could recognise that, say, x**x < 100 implies 0 <= x < 3.59728 (approx), but in general, you can't predict the value of f(x) without actually calculating f(x).
What happens if you accidentally forget to put brackets after the F expression? Do you get a syntax error? Undefined behaviour? A runtime exception?
Steven D'Aprano wrote:
The advantage of F being callable is that it does not need semantic change, the filtering part will be done before expression just like it is right now.
That's not true. It can't be true. If you want to filter on x**x being greater than 100, you need to calculate x**x first. In theory, a sufficiently clever compiler could recognise that, say, x**x < 100 implies 0 <= x < 3.59728 (approx), but in general, you can't predict the value of f(x) without actually calculating f(x).
Did you read the middle part of the post and the diagram, which address the question you're asking and how it would be handled?
What happens if you accidentally forget to put brackets after the F expression? Do you get a syntax error? Undefined behaviour? A runtime exception?
As F is just callable, F > 100 should result in comparison of number against callable. But as it is rare that you really actually wanted to do such thing, I think python can also be a little protective and issue a warning.
On Sun, 21 Jun 2009 04:08:05 am Lie Ryan wrote:
Steven D'Aprano wrote:
The advantage of F being callable is that it does not need semantic change, the filtering part will be done before expression just like it is right now.
That's not true. It can't be true. If you want to filter on x**x being greater than 100, you need to calculate x**x first. In theory, a sufficiently clever compiler could recognise that, say, x**x < 100 implies 0 <= x < 3.59728 (approx), but in general, you can't predict the value of f(x) without actually calculating f(x).
Did you read the middle part of the post and the diagram, which address the question you're asking and how it would be handled?
Yes. It made no sense to me at all.
Let's make a practical example:
c = 1246158315.0 # approximately a week from now L = filter(lambda t: t > c, [time.time() for x in range(20)])
becomes:
L = [time.time() as F for x in range(20) if F() > c]
How do you expect your proposed syntax to determine whether or not the current time is greater than c without actually checking the current time?
Note also that your proposal requires list comps to become like lambda. Using your earlier example:
[x**x as F for x in lst if F() < 100]
This doesn't bind the value of x**x to the name F, but the expression x**x to F. That makes it like a lambda:
lambda x: x**x
except the name x is implied and some sort of magic takes place to ensure that by the time you call F(), the appropriate value of x is still around. If this is to apply to generator expressions as well, calling F() could occur some arbitrarily large time later.
That means that
[time.time() for x in lst]
will create a list that looks something like:
[1245555766.5681751, 1245555767.2609128, ...]
but your proposed:
[time.time() as F for x in lst]
will create a list something like:
[<function F at 0x88a879c>, <function F at 0x88a8d84>, ...]
Steven D'Aprano wrote:
On Sun, 21 Jun 2009 04:08:05 am Lie Ryan wrote:
Steven D'Aprano wrote:
The advantage of F being callable is that it does not need semantic change, the filtering part will be done before expression just like it is right now.
That's not true. It can't be true. If you want to filter on x**x being greater than 100, you need to calculate x**x first. In theory, a sufficiently clever compiler could recognise that, say, x**x < 100 implies 0 <= x < 3.59728 (approx), but in general, you can't predict the value of f(x) without actually calculating f(x).
Did you read the middle part of the post and the diagram, which address the question you're asking and how it would be handled?
Yes. It made no sense to me at all.
Let's make a practical example:
c = 1246158315.0 # approximately a week from now L = filter(lambda t: t > c, [time.time() for x in range(20)])
becomes:
L = [time.time() as F for x in range(20) if F() > c]
How do you expect your proposed syntax to determine whether or not the current time is greater than c without actually checking the current time?
Of course it will call F(). F() will evaluate time.time() and cache it so future call to F() do not need to call time.time() twice (resulting in different time used for filter and expression).
Note also that your proposal requires list comps to become like lambda. Using your earlier example:
[x**x as F for x in lst if F() < 100]
This doesn't bind the value of x**x to the name F, but the expression x**x to F. That makes it like a lambda:
lambda x: x**x
except the name x is implied and some sort of magic takes place to ensure that by the time you call F(), the appropriate value of x is still around. If this is to apply to generator expressions as well, calling F() could occur some arbitrarily large time later.
F() could be called when the filtering takes place or not be called at all, when it does gets called, the expression would be evaluated. In effect this moves forward the expression evaluation to the middle of filtering process.
That means that
[time.time() for x in lst]
will create a list that looks something like:
[1245555766.5681751, 1245555767.2609128, ...]
but your proposed:
[time.time() as F for x in lst]
will create a list something like:
[<function F at 0x88a879c>, <function F at 0x88a8d84>, ...]
Where did you get that idea? The comprehension would call F before appending it to the list. The equivalent for-loop syntax also clearly stated that:
result = [] for x in lst: # F() ensures f(x) will be only ever be called once def F(): nonlocal _cache if not _cache: _cache = f(x) return _cache
_cache = None if g(F()): # here F is called before it's appended, # result is the same as calling f(x) # but prevents evaluating f(x) twice result.append(F())
On Fri, Jun 19, 2009 at 3:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
This, I have wanted.
That doesn't resolve all the issues others raised, but (at least for me), you just moved it from "a little worse than the status quo" to "hmm... that would be nice if it could be done without too many side effect on the rest of the language."
On the possibility that a more general problem sometimes spurs a more elegant solution, I'll point out that I have more often wanted access to the previous or following element, and that my desired filters on the results are often reliant on the results-so-far.
Disadvantages:
- reverses the current semantic of filtering-then-expression.
For What Its Worth, I'm not sure how strong that argument should be. Is this just a bizarre corner case, or is there lots of code that relies on it?
That ordering actually surprises me, because I expect python to evaluate left to right.
On the other hand, now that we have conditional expressions, consistency with those is probably more important, so maybe the number of people surprised by the current situation will go down with time.
-jJ
Jim Jewett jimjjewett@gmail.com writes:
On Fri, Jun 19, 2009 at 3:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
This, I have wanted.
You have it:
res = [f for f in (x**x for x in nums) if f < 100]
In addition to the fact that this works now in existing Python, I find it clearer than the above syntax you say you want.
Ben Finney wrote:
Jim Jewett jimjjewett@gmail.com writes:
On Fri, Jun 19, 2009 at 3:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
This, I have wanted.
You have it:
res = [f for f in (x**x for x in nums) if f < 100]In addition to the fact that this works now in existing Python, I find it clearer than the above syntax you say you want.
How about:
res = [F for x in nums with x**x as F if F < 100]
:-)
On Sat, Jun 20, 2009 at 9:22 AM, MRABpython@mrabarnett.plus.com wrote:
Ben Finney wrote:
Jim Jewett jimjjewett@gmail.com writes:
On Fri, Jun 19, 2009 at 3:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
This, I have wanted.
You have it:
res = [f for f in (x**x for x in nums) if f < 100]
In addition to the fact that this works now in existing Python, I find it clearer than the above syntax you say you want.
How about:
res = [F for x in nums with x**x as F if F < 100]
:-)
That toggles the first part of the comprehensions to be or not be an expression, depending on if there is a with clause later. You could miss this when you read it, and it opens the door to doing more strange things, like:
res = [F/2 for x in nums with x**x as F if F < 100]
This is basically a strangely syntaxed nested loop
Calvin Spealman wrote:
On Sat, Jun 20, 2009 at 9:22 AM, MRABpython@mrabarnett.plus.com wrote:
Ben Finney wrote:
Jim Jewett jimjjewett@gmail.com writes:
On Fri, Jun 19, 2009 at 3:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
This, I have wanted.
You have it:
res = [f for f in (x**x for x in nums) if f < 100]
In addition to the fact that this works now in existing Python, I find it clearer than the above syntax you say you want.
How about:
res = [F for x in nums with x**x as F if F < 100]
:-)
That toggles the first part of the comprehensions to be or not be an expression, depending on if there is a with clause later. You could miss this when you read it, and it opens the door to doing more strange things, like:
res = [F/2 for x in nums with x**x as F if F < 100]
This is basically a strangely syntaxed nested loop
I don't know what you mean.
def F(m, x):
print m, x return x
[foo("value", x) for x in range(5) if foo("test", x) % 2]
test 0 test 1 value 1 test 2 test 3 value 3 test 4 [1, 3]
The test is done first, so it's equivalent to:
results = [] for x in range(5):
if foo("test", x) % 2: results.append(foo("value", x))
test 0 test 1 value 1 test 2 test 3 value 3 test 4
That means that:
res = [x**x for x in nums if x**x < 100]
is equivalent to:
res = [] for x in nums : if x**x < 100: res.append(x**x)
My suggestions turns:
res = [] for x in nums : F = x**x # <== temporary variable if F < 100: res.append(F)
into:
res = [F for x in nums with x**x as F if F < 100] ^^^^^^^^^^^^^^ temporary variable
On Fri, Jun 19, 2009 at 2:39 AM, Lie Ryanlie.1296@gmail.com wrote:
res = [x**x as F for x in nums if F < 100]
In some languages, you can use "let" in a list comprehension:
[f | x <- nums, let f = x**x, f < 100]
which I guess might look something like this in python:
[f for x in nums let f = x**x if f < 100]
(JavaScript actually has a `let` keyword so I suspect we will eventually adopt something like that.)
-j
On Fri, 19 Jun 2009 05:39:32 pm Lie Ryan wrote:
Aahz wrote:
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
I think of it not as limitation but as an odd gap in functionality. I think having the semantics that the filtering is done after the expression part would be much more useful than the current behavior (filtering before expression).
The point of the filtering is to avoid needlessly calculating a potentially expensive expression only to throw it away.
If you want expression first, then filter, you can get that already in a one-liner:
filter(lambda x: x > 0, [f(x) for x in seq])
Don't create new syntax when there are perfectly good functions that do the job already.
Steven D'Aprano wrote:
On Fri, 19 Jun 2009 05:39:32 pm Lie Ryan wrote:
Aahz wrote:
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
I think of it not as limitation but as an odd gap in functionality. I think having the semantics that the filtering is done after the expression part would be much more useful than the current behavior (filtering before expression).
The point of the filtering is to avoid needlessly calculating a potentially expensive expression only to throw it away.
If you want expression first, then filter, you can get that already in a one-liner:
filter(lambda x: x > 0, [f(x) for x in seq])
Don't create new syntax when there are perfectly good functions that do the job already.
That's ugly because of the same reason for using map(): [y for y in map(lambda x: f(x), seq) if y > 0]
or nested comprehension: [y for y in (f(x) for x in seq) if y > 0]
On Sat, 20 Jun 2009 08:31:26 am Lie Ryan wrote:
If you want expression first, then filter, you can get that already in a one-liner:
filter(lambda x: x > 0, [f(x) for x in seq])
Don't create new syntax when there are perfectly good functions that do the job already.
That's ugly because of the same reason for using map(): [y for y in map(lambda x: f(x), seq) if y > 0]
You don't like lambda? Fine, define an external function first. Then you can write:
filter(pred, (f(x) for x in seq))
There's no violation of DRY, there's no redundancy, there's no lambda, there's no "y" variable needed. What's ugly about it?
or nested comprehension: [y for y in (f(x) for x in seq) if y > 0]
You seem to be labouring under the misapprehension that anything that requires two steps instead of one is "ugly". I find the nested comprehension perfectly readable, although for more complicated cases I'd split it into two explicit steps. It is (almost) completely general, covering both filtering on *both* input args and output args:
gen = (3*x**2-5*x+4 for x in seq if x % 3 != 2) result = [y for y in gen if -3 < y < 3]
The only case it doesn't cover is where the second filter depends on the value of x, and even that can be covered with a *tiny* bit more work:
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [y[1] for y in gen if -y[0] < y[1] < y[0]]
It requires no new syntax, no changes to the behaviour of list comps, no new meaning on "as", it's understandable and readable.
Compare your suggestion:
[3*x**2-5*x+4 as y for x in seq if (x % 3 != 2) and (-x < y < x)]
Disadvantages:
- It requires new syntax. - It requires new behaviour to list comps and generator expressions. - It creates yet another meaning to the keyword "as".
Advantages:
- It requires no intermediate tuple. But since intermediate tuples are only required for a tiny proportion of use-cases, this is not much of a advantage. - It loops over the data once rather than twice, but since it does twice as much work inside the loop the only saving is the setup and teardown costs of the second loop. Truly a micro-optimization, and premature at that.
Steven D'Aprano wrote: <snip>
The only case it doesn't cover is where the second filter depends on the value of x, and even that can be covered with a *tiny* bit more work:
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [y[1] for y in gen if -y[0] < y[1] < y[0]]
It requires no new syntax, no changes to the behaviour of list comps, no new meaning on "as", it's understandable and readable.
Compare your suggestion:
[3*x**2-5*x+4 as y for x in seq if (x % 3 != 2) and (-x < y < x)]
For me, this one is much clearer, understandable, and readable than y[0] and y[1]; and you still have the option to split them if you think y[0] and y[1] is better.
Disadvantages:
- It requires new syntax.
- It requires new behaviour to list comps and generator expressions.
- It creates yet another meaning to the keyword "as".
"as" is already used for renaming keyword in "with" and "import" statement, so I don't think it actually creates any more meaning that we don't already have.
Lie Ryan lie.1296@gmail.com writes:
Steven D'Aprano wrote:
<snip> > The only case it doesn't cover is where the second filter depends on > the value of x, and even that can be covered with a *tiny* bit more > work: > > gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) > result = [y[1] for y in gen if -y[0] < y[1] < y[0]] > > It requires no new syntax, no changes to the behaviour of list > comps, no new meaning on "as", it's understandable and readable.
I think it would be more readable without index references, but instead using tuple unpacking::
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [b for (a, b) in gen if -a < b < a]
It can even be done as a single expression without (IMO) significantly affecting readability::
result = [ b for (a, b) in ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) if -a < b < a]
Compare your suggestion:
[3*x**2-5*x+4 as y for x in seq if (x % 3 != 2) and (-x < y < x)]
For me, this one is much clearer, understandable, and readable than y[0] and y[1]; and you still have the option to split them if you think y[0] and y[1] is better.
I hope you'll agree that my above suggestion retains this, without needing any new syntax.
Ben Finney wrote:
Lie Ryan lie.1296@gmail.com writes:
Steven D'Aprano wrote:
<snip> > The only case it doesn't cover is where the second filter depends on > the value of x, and even that can be covered with a *tiny* bit more > work: > > gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) > result = [y[1] for y in gen if -y[0] < y[1] < y[0]] > > It requires no new syntax, no changes to the behaviour of list > comps, no new meaning on "as", it's understandable and readable.
I think it would be more readable without index references, but instead using tuple unpacking::
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [b for (a, b) in gen if -a < b < a]It can even be done as a single expression without (IMO) significantly affecting readability::
result = [ b for (a, b) in ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) if -a < b < a]
IMHO, when a comprehension requires more than a single line, it should turn into explicit loop.
<snip>
What about where/let expression? [(y, y) for x in some_list if y < 0 where y = f(x)]
On Sun, Jun 21, 2009 at 10:36 AM, Lie Ryanlie.1296@gmail.com wrote:
Ben Finney wrote:
Lie Ryan lie.1296@gmail.com writes:
Steven D'Aprano wrote:
<snip> > The only case it doesn't cover is where the second filter depends on > the value of x, and even that can be covered with a *tiny* bit more > work: > > gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) > result = [y[1] for y in gen if -y[0] < y[1] < y[0]] > > It requires no new syntax, no changes to the behaviour of list > comps, no new meaning on "as", it's understandable and readable.
I think it would be more readable without index references, but instead using tuple unpacking::
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [b for (a, b) in gen if -a < b < a]
It can even be done as a single expression without (IMO) significantly affecting readability::
result = [ b for (a, b) in ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) if -a < b < a]
IMHO, when a comprehension requires more than a single line, it should turn into explicit loop.
<snip>
Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
On Sun, Jun 21, 2009 at 10:36 AM, Lie Ryanlie.1296@gmail.com wrote:
Ben Finney wrote:
Lie Ryan lie.1296@gmail.com writes:
Steven D'Aprano wrote:
<snip> > The only case it doesn't cover is where the second filter depends on > the value of x, and even that can be covered with a *tiny* bit more > work: > > gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) > result = [y[1] for y in gen if -y[0] < y[1] < y[0]] > > It requires no new syntax, no changes to the behaviour of list > comps, no new meaning on "as", it's understandable and readable.
I think it would be more readable without index references, but instead using tuple unpacking::
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) result = [b for (a, b) in gen if -a < b < a]
It can even be done as a single expression without (IMO) significantly affecting readability::
result = [ b for (a, b) in ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) if -a < b < a]
IMHO, when a comprehension requires more than a single line, it should turn into explicit loop.
<snip>
On Sun, Jun 21, 2009 at 1:54 AM, Andrey Popp8mayday@gmail.com wrote:
What about where/let expression? [(y, y) for x in some_list if y < 0 where y = f(x)] -- С уважением, Андрей Попп. +7 911 740 24 91
(A) Please don't top-post. (http://en.wikipedia.org/wiki/Top-post) (B) That has the distinct disadvantage of adding a new keyword. I instead prefer the "as" version of the proposal for this reason.
Cheers, Chris
On Jun 21, 2009 1:06pm, Chris Rebert pyideas@rebertia.com wrote:
On Sun, Jun 21, 2009 at 10:36 AM, Lie Ryanlie.1296@gmail.com> wrote:
Ben Finney wrote:
Lie Ryan lie.1296@gmail.com> writes:
Steven D'Aprano wrote:
The only case it doesn't cover is where the second filter depends on
the value of x, and even that can be covered with a *tiny* bit more
work:
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2)
result = [y[1] for y in gen if -y[0]
It requires no new syntax, no changes to the behaviour of list
comps, no new meaning on "as", it's understandable and readable.
I think it would be more readable without index references, but
instead
using tuple unpacking::
gen = ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2)
result = [b for (a, b) in gen if -a
It can even be done as a single expression without (IMO) significantly
affecting readability::
result = [
b for (a, b) in
((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2)
if -a
IMHO, when a comprehension requires more than a single line, it should
turn into explicit loop.
On Sun, Jun 21, 2009 at 1:54 AM, Andrey Popp8mayday@gmail.com> wrote:
What about where/let expression?
[(y, y) for x in some_list if y
С уважением, Андрей Попп.
+7 911 740 24 91
(A) Please don't top-post. (http://en.wikipedia.org/wiki/Top-post)
(B) That has the distinct disadvantage of adding a new keyword. I
instead prefer the "as" version of the proposal for this reason.
Cheers,
Chris
--
(A) Sorry (B) There is no difference, except "where" is widely used in othe languages. Anyway, I think that "where" like functionality would be useful in list comprehensions, lambdas...
2009/6/21 Lie Ryan lie.1296@gmail.com:
It can even be done as a single expression without (IMO) significantly affecting readability::
result = [ b for (a, b) in ((x, 3*x**2-5*x+4) for x in seq if x % 3 != 2) if -a < b < a]
IMHO, when a comprehension requires more than a single line, it should turn into explicit loop.
Hang on - isn't the whole point of this thread that, in your opinion, when a comprehension gets complex, it requires new syntax? :-)
Paul.
Steven D'Aprano wrote:
On Sat, 20 Jun 2009 08:31:26 am Lie Ryan wrote:
You don't like lambda? Fine, define an external function first. Then you can write:
filter(pred, (f(x) for x in seq))
There's no violation of DRY, there's no redundancy, there's no lambda, there's no "y" variable needed. What's ugly about it?
I think its great and that it kills any justification for the proposal.
tjr
Terry Reedy wrote:
Steven D'Aprano wrote:
On Sat, 20 Jun 2009 08:31:26 am Lie Ryan wrote:
You don't like lambda? Fine, define an external function first. Then you can write:
filter(pred, (f(x) for x in seq))
There's no violation of DRY, there's no redundancy, there's no lambda, there's no "y" variable needed. What's ugly about it?
I think its great and that it kills any justification for the proposal.
tjr
I hate it. It mixes map/filter style and comprehension style; and the fact it does so in a single line only makes it worse. Not that it would be any better in two lines.
Lie Ryan schrieb:
Terry Reedy wrote:
Steven D'Aprano wrote:
On Sat, 20 Jun 2009 08:31:26 am Lie Ryan wrote:
You don't like lambda? Fine, define an external function first. Then you can write:
filter(pred, (f(x) for x in seq))
There's no violation of DRY, there's no redundancy, there's no lambda, there's no "y" variable needed. What's ugly about it?
I think its great and that it kills any justification for the proposal.
tjr
I hate it. It mixes map/filter style and comprehension style; and the fact it does so in a single line only makes it worse. Not that it would be any better in two lines.
Taking this further, using only map/filter style like this
filter(pred, map(f, seq))
takes two steps. Why is it so bad that doing it in a listcomp
(el for el in (f(y) for y in seq) if el > 2)
takes two steps as well?
Georg
Georg Brandl wrote:
Lie Ryan schrieb:
Terry Reedy wrote:
Steven D'Aprano wrote:
On Sat, 20 Jun 2009 08:31:26 am Lie Ryan wrote: You don't like lambda? Fine, define an external function first. Then you can write:
filter(pred, (f(x) for x in seq))
There's no violation of DRY, there's no redundancy, there's no lambda, there's no "y" variable needed. What's ugly about it?
I think its great and that it kills any justification for the proposal.
tjr
I hate it. It mixes map/filter style and comprehension style; and the fact it does so in a single line only makes it worse. Not that it would be any better in two lines.
Taking this further, using only map/filter style like this
filter(pred, map(f, seq))
takes two steps. Why is it so bad that doing it in a listcomp
(el for el in (f(y) for y in seq) if el > 2)
takes two steps as well?
Georg
Comprehension, by its nature, is map+filter in a single expression. Nested comprehension is (map+filter)+(map+filter).
(where + is some sort of function composition)
The proposal enables comprehension to become filter+map+filter, map+filter, filter+map, or map-only; eliminating the redundant map in (map+filter)+(map+filter).
The filter() and map() functions are two separate function in the first place, so there is no redundancy in it.
Mixing the two styles is ugly since it means you have to think in two separate (though related) paradigms.
Lie Ryan writes:
The proposal enables comprehension to become filter+map+filter, map+filter, filter+map, or map-only; eliminating the redundant map in (map+filter)+(map+filter).
The problem with the proposal is that that is *all* it does. I don't see how you plan to disambiguate in cases where the desired operation "really is" (map+filter)+(map+filter). So what you're stuck at is "I want this one operation to be a one-liner." The reply to that is "not every two-line function needs to have special syntax."
Unless you can generalize this proposal to handle more general lambdas in a beautiful and Pythonic way, you've already been offered two obvious and nice ways to do it, and another obvious way that is an "ugly" mixed metaphor. All of the other idioms *do* generalize. We just spent several years de-cluttering Python 3, it's too soon to start cluttering it up again, no matter how beautiful it makes this special case.
And I happen to think Ben's iterated generator comprehension is the most beautiful of the lot. YMMV of course but so does a that of a lot of other people; getting a consensus for new syntax is going to be impossible if you don't generalize the proposal.
Stephen J. Turnbull wrote:
Lie Ryan writes:
The proposal enables comprehension to become filter+map+filter, map+filter, filter+map, or map-only; eliminating the redundant map in (map+filter)+(map+filter).
The problem with the proposal is that that is *all* it does. I don't see how you plan to disambiguate in cases where the desired operation "really is" (map+filter)+(map+filter). So what you're stuck at is "I want this one operation to be a one-liner." The reply to that is "not every two-line function needs to have special syntax."
The regular map+filter is still available by not using the `as` keyword (or by not calling F). Thus when what you really wanted is two maps, you need to use two comprehensions. The number of maps should == The number of comprehension; currently it is not always possible to do so, when the assumed map and filter ordering doesn't match our required ordering.
Lie Ryan writes:
The regular map+filter is still available by not using the `as` keyword (or by not calling F). Thus when what you really wanted is two maps, you need to use two comprehensions. The number of maps should == The number of comprehension; currently it is not always possible to do so, when the assumed map and filter ordering doesn't match our required ordering.
I understand that; the first part is a trivial consequence of backward compatibility, and the second you've been at pains to explain already. I'm not trying to say you're *wrong*. The appeal of the proposed syntax in the particular case is obvious.
Admittedly, I personally don't find it particularly useful. I don't have any problem at all with decomposing what you consider to be a single comprehension into a pipeline of generators. It's efficient and elegant, and it's not clear to me that your construct can generate better byte code than Ben's nested comprehensions. If not, your claim that "this is conceptually a single comprehension, why break it into two or more" seems to me to be founded on quicksand. But that's beside the point, I don't have to like all the constructs that other people find useful. Even Guido has allowed things into Python that he personally dislikes quite a bit.
I'm saying it's my impression that it will be *insufficient*. It's like if somebody suggested introducing a unary operator "**" to denote "squared". That's just not useful enough, while a *binary* operator "**" for "power" is useful enough to have been added ages ago.
In other words, that kind of logic hasn't been able to justify *proliferation* of syntax in the more than 5 years I've been following Python-Dev (and now Python-Ideas). It only works when the new syntax is sufficiently comprehensive to replace "all the old uglies" in some sense. Even if you can't clear the hurdle in one bound, you need to aim at something like "getting rid of *all* lambda/external function definitions in comprehensions".
Lie Ryan schrieb:
I hate it. It mixes map/filter style and comprehension style; and the fact it does so in a single line only makes it worse. Not that it would be any better in two lines.
Taking this further, using only map/filter style like this
filter(pred, map(f, seq))
takes two steps. Why is it so bad that doing it in a listcomp
(el for el in (f(y) for y in seq) if el > 2)
takes two steps as well?
Georg
Comprehension, by its nature, is map+filter in a single expression.
Let's stick with "+" for function composition. Then comprehension is, as you say, ``map + filter`` (map after filter). It is *not* ``filter + map`` (filter after map), which is what would be needed in the case discussed in this thread. In the functional waym you can of course combine map and filter in the order you like.
Nested comprehension is (map+filter)+(map+filter).
Yes, and in this case, we use it as ``(id+filter) + (map+id)``, which is the easiest way to build a ``filter + map`` with the ``map + filter`` building block.
Mixing the two styles is ugly since it means you have to think in two separate (though related) paradigms.
I mentally translate map and filter into comprehensions anyway (or maybe rather both translate into the same more abstract thing), so I don't see it as ugly.
Georg
Lie Ryan schrieb:
Steven D'Aprano wrote:
On Fri, 19 Jun 2009 05:39:32 pm Lie Ryan wrote:
Aahz wrote:
On Fri, Jun 19, 2009, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Listcomps and genexps are like lambdas: run up against their limits and you should switch to a regular for loop or generator.
I think of it not as limitation but as an odd gap in functionality. I think having the semantics that the filtering is done after the expression part would be much more useful than the current behavior (filtering before expression).
The point of the filtering is to avoid needlessly calculating a potentially expensive expression only to throw it away.
If you want expression first, then filter, you can get that already in a one-liner:
filter(lambda x: x > 0, [f(x) for x in seq])
Don't create new syntax when there are perfectly good functions that do the job already.
That's ugly because of the same reason for using map(): [y for y in map(lambda x: f(x), seq) if y > 0]
Especially if f is already a handy callable of one argument, no need to use a lambda.
Georg
Georg Brandl wrote:
Lie Ryan schrieb:
That's ugly because of the same reason for using map(): [y for y in map(lambda x: f(x), seq) if y > 0]
Especially if f is already a handy callable of one argument, no need to use a lambda.
Ahh... yes. Sometimes I actually meant f(x) to be a substitute to any expression (both simple: x**2 or complex: foo(x))
but well, I didn't actually see that one...
2009/6/18 Lie Ryan lie.1296@gmail.com:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
[y for y in (f(x) for x in l) if y > 0]
But as Aahz says, if it's too complex, use an explicit loop. Not everything needs to be a one-liner.
Paul
Paul Moore wrote:
2009/6/18 Lie Ryan lie.1296@gmail.com:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
[y for y in (f(x) for x in l) if y > 0]
But as Aahz says, if it's too complex, use an explicit loop. Not everything needs to be a one-liner.
Paul
Avoiding a nested list comprehension is exactly the motivation for the idea.
[Lie Ryan]
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
[x for x in map(f, l) if x > 0]
Raymond
Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Even if f() is not a function such as
[x + 1 for x in l if x + 1 > 0]
it looks ugly since we're repeating ourself.
We can work around it like this:
[y for y in (f(x) for x in l) if y > 0]
but then we have an unnecessary nested loop comprehension.
I'm suggesting about something like:
[f(x) for x in l if @ > 0]
where @ is the result of the listcomp's expression (i.e. f(x))
Personally, I don't like the use of symbols like @, as python is not perl. I'm still thinking of a better syntax/approach and is open for suggestion.
What do you guys think?
IMO this is premature optimization. (I know that's an annoying thing to say, but bear with me...)
Just using [f(x) for x in nums if f(x) > 0] is the most readable and obvious option. I think the case where such a line of code actually needs to be optimized is rare, and on there rare occasions using a slightly less readable variant is reasonable (along with an insightful comment).
In other words, from my experience all we would gain from the proposed new syntax is making such premature optimization easier, at the cost of less readable code and more complex syntax.
- Tal
On Fri, Jun 19, 2009 at 7:34 AM, Tal Einattaleinat@gmail.com wrote:
Just using [f(x) for x in nums if f(x) > 0] is the most readable and obvious option.
I would often prefer to break it into two steps:
temp = (f(x) for x in nums) results = [e for e in temp if e>0]
Others will dislike the extra line and temp var, which is one reason it isn't among the several solutions previously suggested. Are these differences big enough (or the solutions obscure enough) that the variation is itself a cost of the current situation?
-jJ
On Fri, Jun 19, 2009 at 4:45 PM, Jim Jewettjimjjewett@gmail.com wrote:
On Fri, Jun 19, 2009 at 7:34 AM, Tal Einattaleinat@gmail.com wrote:
Just using [f(x) for x in nums if f(x) > 0] is the most readable and obvious option.
I would often prefer to break it into two steps:
temp = (f(x) for x in nums) results = [e for e in temp if e>0]
I sometimes find myself doing that too. That's probably more readable than what I wrote :)
Others will dislike the extra line and temp var, which is one reason it isn't among the several solutions previously suggested.
I sometimes avoid the temporary variable by using the same one twice, e.g.: results = (f(x) for x in nums) results = [res for res in results if res > 0]
In the above example the second line is just filtering the results, and I feel this idiom conveys the idea pretty well.
- Tal
Le Fri, 19 Jun 2009 09:45:08 -0400, Jim Jewett jimjjewett@gmail.com s'exprima ainsi:
On Fri, Jun 19, 2009 at 7:34 AM, Tal Einattaleinat@gmail.com wrote:
Just using [f(x) for x in nums if f(x) > 0] is the most readable and obvious option.
I would often prefer to break it into two steps:
temp = (f(x) for x in nums) results = [e for e in temp if e>0]Others will dislike the extra line and temp var, which is one reason it isn't among the several solutions previously suggested. Are these differences big enough (or the solutions obscure enough) that the variation is itself a cost of the current situation?
Ditto for me.
A list comp with both computation and filtering is for me two ideas, two steps, so I write two lines anyway. This also has the advantage to disambiguate the order issue (computation or filter first?) and there is no risk of double side-effect (which anyway I never use, but who knows...).
Denis ------ la vita e estrany
You could write: [x|y <- l, x <- [f(y)], x > 0]
Oh, wait. Thats Haskell. And even in haskell you would write: [x|x <- map f l, x > 0]
In Python you can write: [x for x in map(f,l) if x > 0]
In Python 2.x you may want to write: from itertools import imap [x for x in imap(f,l) if x > 0]
A more SQL like approach that would fit somewhat with pythons syntax would be (as you can see its exactly the same lengths as the above but needs a new name): [f(x) as y for x in l if y > 0]
Because in SQL you can write (IIRC): select f(x) as y from l where y > 0;
Maybe something like .Nets LINQ would be a nice idea to integrate in python?
-panzi
Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Even if f() is not a function such as
[x + 1 for x in l if x + 1 > 0]
it looks ugly since we're repeating ourself.
We can work around it like this:
[y for y in (f(x) for x in l) if y > 0]
but then we have an unnecessary nested loop comprehension.
I'm suggesting about something like:
[f(x) for x in l if @ > 0]
where @ is the result of the listcomp's expression (i.e. f(x))
Personally, I don't like the use of symbols like @, as python is not perl. I'm still thinking of a better syntax/approach and is open for suggestion.
What do you guys think?
On Fri, Jun 19, 2009 at 10:54 AM, Mathias Panzenböckgrosser.meister.morti@gmx.net wrote:
You could write: [x|y <- l, x <- [f(y)], x > 0]
Oh, wait. Thats Haskell. And even in haskell you would write: [x|x <- map f l, x > 0]
In Python you can write: [x for x in map(f,l) if x > 0]
In Python 2.x you may want to write: from itertools import imap [x for x in imap(f,l) if x > 0]
A more SQL like approach that would fit somewhat with pythons syntax would be (as you can see its exactly the same lengths as the above but needs a new name): [f(x) as y for x in l if y > 0]
Because in SQL you can write (IIRC): select f(x) as y from l where y > 0;
Maybe something like .Nets LINQ would be a nice idea to integrate in python?
Comprehensions and generator expressions already give us most of the LINQ functionality. Add in `list()` and the ability to `.sort()` lists with a `key` argument and you have the entire thing, except for the one corner case being discussed. Unless I've overlooked something...
Cheers, Chris
Chris Rebert wrote:
On Fri, Jun 19, 2009 at 10:54 AM, Mathias Panzenböckgrosser.meister.morti@gmx.net wrote:
You could write: [x|y <- l, x <- [f(y)], x > 0]
Oh, wait. Thats Haskell. And even in haskell you would write: [x|x <- map f l, x > 0]
In Python you can write: [x for x in map(f,l) if x > 0]
In Python 2.x you may want to write: from itertools import imap [x for x in imap(f,l) if x > 0]
A more SQL like approach that would fit somewhat with pythons syntax would be (as you can see its exactly the same lengths as the above but needs a new name): [f(x) as y for x in l if y > 0]
Because in SQL you can write (IIRC): select f(x) as y from l where y > 0;
Maybe something like .Nets LINQ would be a nice idea to integrate in python?
Comprehensions and generator expressions already give us most of the LINQ functionality. Add in `list()` and the ability to `.sort()` lists with a `key` argument and you have the entire thing, except for the one corner case being discussed. Unless I've overlooked something...
Yes: With LINQ its possible to build a query object out of an LINQ expression instead of evaluating it eagerly. This is used primarily to generate SQL code while still using syntax native to the host language (C#) and preserving type safety (ok the later cannot be done in python).
-panzi
On Fri, Jun 19, 2009 at 12:56 PM, Mathias Panzenböckgrosser.meister.morti@gmx.net wrote:
Chris Rebert wrote:
On Fri, Jun 19, 2009 at 10:54 AM, Mathias Panzenböckgrosser.meister.morti@gmx.net wrote:
You could write: [x|y <- l, x <- [f(y)], x > 0]
Oh, wait. Thats Haskell. And even in haskell you would write: [x|x <- map f l, x > 0]
In Python you can write: [x for x in map(f,l) if x > 0]
In Python 2.x you may want to write: from itertools import imap [x for x in imap(f,l) if x > 0]
A more SQL like approach that would fit somewhat with pythons syntax would be (as you can see its exactly the same lengths as the above but needs a new name): [f(x) as y for x in l if y > 0]
Because in SQL you can write (IIRC): select f(x) as y from l where y > 0;
Maybe something like .Nets LINQ would be a nice idea to integrate in python?
Comprehensions and generator expressions already give us most of the LINQ functionality. Add in `list()` and the ability to `.sort()` lists with a `key` argument and you have the entire thing, except for the one corner case being discussed. Unless I've overlooked something...
Yes: With LINQ its possible to build a query object out of an LINQ expression instead of evaluating it eagerly. This is used primarily to generate SQL code while still using syntax native to the host language (C#) and preserving type safety (ok the later cannot be done in python).
One could probably hack that part together with lambdas, the ast module, and some black magic though. And are there any use cases besides SQL? But point taken.
Cheers, Chris
Chris Rebert wrote:
On Fri, Jun 19, 2009 at 12:56 PM, Mathias Panzenböckgrosser.meister.morti@gmx.net wrote:
Chris Rebert wrote:
Comprehensions and generator expressions already give us most of the LINQ functionality. Add in `list()` and the ability to `.sort()` lists with a `key` argument and you have the entire thing, except for the one corner case being discussed. Unless I've overlooked something...
Yes: With LINQ its possible to build a query object out of an LINQ expression instead of evaluating it eagerly. This is used primarily to generate SQL code while still using syntax native to the host language (C#) and preserving type safety (ok the later cannot be done in python).
One could probably hack that part together with lambdas, the ast module, and some black magic though. And are there any use cases besides SQL?
Yes: Queries on XML data. So you have the exact same Syntax for queries on simple lists, SQL databases and XML files (but yes, using LINGQ for XML is still much more to write than using something like XPath). I think you can also add your own backends if you like (e.g. for yaml?). And it's all native syntax (no limits on expressiveness and no problems concerning string escaping etc.).
-panzi
On 18 Jun 2009, at 21:19, Lie Ryan wrote:
In list/generator comprehension, currently we have no way to access the result of the expression and have to write something like this:
[f(x) for x in l if f(x) > 0]
if f() is heavy or non-pure (i.e. have side effects), calling f() twice might be undesirable.
Even if f() is not a function such as
[x + 1 for x in l if x + 1 > 0]
it looks ugly since we're repeating ourself.
We can work around it like this:
[y for y in (f(x) for x in l) if y > 0]
but then we have an unnecessary nested loop comprehension
You can write:
[y for x in l for y in [f(x)] if y > 0]
Which is even worse ;)
participants (20)
-
8mayday＠gmail.com -
Aahz -
Andrey Popp
-
Arnaud Delobelle -
Ben Finney -
Calvin Spealman -
Chris Rebert -
Georg Brandl -
Jason Orendorff -
Jim Jewett -
Lie Ryan -
Mathias Panzenböck -
MRAB -
Paul Moore -
Raymond Hettinger -
spir -
Stephen J. Turnbull -
Steven D'Aprano -
Tal Einat -
Terry Reedy