[Python-ideas] With clauses for generator expressions

[Andrew Barnert]

From: Serhiy Storchaka <storchaka at gmail.com>
Sent: Thu, November 15, 2012 4:17:42 AM


> On 15.11.12 13:11, Andrew Barnert wrote:
> > Which means the only question  is, which one looks more readable:
> > 
> > 1. (foo(line) for line in  baz(f) if 'bar' in line with open('foo') as f)
> > 2. (foo(line) for line in  baz(f) with open('foo') as f if 'bar' in line)
> > 3. (foo(line) with  open('foo') as f for line in baz(f) if 'bar' in line)
> 
> What about such  generator?
> 
> def gen():
>     with a() as f:
>          for x in f:
>             if  p(x):
>                 with b(x) as  g:
>                     for  y in g:
>                          if q(y):
>                              yield  y


Mechanically transforming that is easy. You just insert each with along with its 
corresponding for and if. There are no ambiguities for any of the three 
potential rules:

1. (y for x in f if p(x) with a() as f for y in g if q(y) with b(x) as g)
2. (y for x in f with a() as f if p(x) for y in g with b(x) as g if q(y))
3. (y with a() as f for x in f if p(x) with b(x) as g for y in g if q(y))

I suppose you could also argue for a "super-1" where we stick all the withs at 
the end, or a "super-3" where we stick them all at the beginning… but it's hard 
to see a compelling argument for that. In fact, despite everything I said about 
clause structure not implying nesting, either one of those would look to me as 
if all the withs were at the outermost scope.

At any rate, unlike the simpler cases, here I have no opinion on which of those 
is clearest. They're all impossible to read at a glance (although breaking them 
up into multiple lines helps, I still don't have any clue what even the original 
function means—all those one-letter variables and functions, with 
easily-confused letters to boot…). But they're all quite easy to work out, or to 
construct, if you understand nested generator expressions and know the rule for 
where each clause goes.