[Python-ideas] FW: Map-then-filter in comprehensions

[Brendan Barnwell]

On 2016-03-08 11:54, Chris Angelico wrote:
> On Wed, Mar 9, 2016 at 6:51 AM, Ethan Furman<ethan at stoneleaf.us>  wrote:
>> >So?  Enhance the idea to be allowing
>> >
>> >(expr) as (name) anywhere:
>> >
>> >   if (file.read() as data):
>> >       # process data
>> >
>> >I'm cool with that.  :)
> Well, okay then. Spin that off as its own idea: Name bindings in
> expressions. It's a coherent idea, but it's likely to see some fairly
> stiff opposition :)
>
> (For what it's worth, I am_not_  in opposition to it.)

	I agree that that option should be considered.  I don't see that much 
value in the current proposals because they only handle the case where 
you want to filter the returned values based on those same values, but 
not the case where you want to filter (before or after the map) based on 
some criterion that re-uses a computed expression, or where you want to 
re-use an expression that's not in a comprehension at all.  Something like:

[some_dict[x.lower()] for x in whatever if x.lower() not in 
exclusion_list and x.lower() in some_dict]

	Even though this filter is being done before the map, it's still 
awkward, because you have to keep repeating the .lower().  If we had 
assignment-as-expression, you could do

[some_dict[lx] for x in whatever if lx not in exclusion_list and lx in 
some_dict where lx=x.lower()]

(or whatever the syntax may be).

	Of course, that example is rather silly since the version with the 
"where" is barely shorter :-).  But hopefully the point is clear.  I 
often find myself doing awkward things like the first example in 
numerical situations where I want to collect one thing while filtering 
and mapping based on something else (e.g., work with numbers while 
filtering and mapping based on their logs or something).

	Moreover, a general-purpose assignment-as-expression would also be 
usable in other contexts besides comprehensions.  I run into this same 
repeated-expression thing when doing computations in pandas using 
.apply(), where you do .apply(lambda x: ...), and maybe the ... involves 
a repeated expression involving x.  With assignment-as-expression you 
could .apply(lambda x: ... where temp=f(x)) or what have you

	I think adding assignment-as-expression would be a significant change 
to Python.  I'm not sure whether it would overall be a good change. 
Almost all the cases where I really feel like I want this are cases 
where I'm working with data interactively, and it really is easier to 
use a lambda than define a separate one-off function.  For anything 
larger-scale, I agree that the "problem" can be solved by taking a deep 
breath and writing a separate function or generator comprehension to do 
what you want.  As someone mentioned earlier on this thread, the 
original motivating example:

foo = [abs(x) for x in numbers if abs(x) > 5]

already has a crystal clear solution:

abses = (abs(x) for x in numbers)
foo = [x for x in abses if x > 5]

There is really nothing wrong with this existing solution unless you're 
working interactively and want to do things more tersely.  But if you're 
doing that, you're likely to want to do other kinds of terse expressions 
besides list comprehensions too.

	So basically, to me the issue is much more general than "I want to 
filter after mapping in comprehensions".  What I want is to write 
expressions that re-use an intermediate computation.  Some of those are 
in comprehensions, some aren't.  Most of the cases where I want to do 
this are in interactive situations where I'm using a lot of one-off 
lambdas and temp expressions.  If that's something we want to support 
more, assignment-as-expression could be quite useful, but if we don't, 
it's a different story.

-- 
Brendan Barnwell
"Do not follow where the path may lead.  Go, instead, where there is no 
path, and leave a trail."
    --author unknown