[Python-ideas] With clauses for generator expressions

[Terry Reedy]

On 11/16/2012 4:09 AM, Andrew Barnert wrote:
> So far, nearly everyone is discussing things which are tangential, or arguing
> that one of the optional variants is bad. So let me strip down the proposal,
> without any options in it, and expand on a use case. The syntax is:
>
>
>      (foo(line) with open('bar') as f for line in baz(f))

OK, that's helpful. Now let me strip down my objection to this: your 
proposal is conceptually wrong because it mixes two distinct and 
different ideas -- collection definition and context management. It 
conflicts with a well-defined notion of long standing.

To explain: in math, one can define a set explicitly by displaying the 
members or implicitly as a subset of based on one or more base sets. 
Using one version of the notation
{0, 2, 4} == {2*i| i in N; i < 3}
The latter is 'set-builder notation' or a 'set comprehension' (and would 
usually use the epsilon-like member symbol instead of 'in'). The idea 
goes back at least a century.
https://en.wikipedia.org/wiki/Set-builder_notation

In Python, the latter directly translates to
   {2*i for i in itertools.count() if i < 3} ==
   {i for i in range(0, 5, 2)}
(Python does not require the base collection to match the result class.)
Another pair of examples:
   {(i,j)| i in N, j in N; i+j <= 5}
   {(i,j) for i in count() for j in count if i+j <= 5}

Similar usage in programming go back over half a century.
https://en.wikipedia.org/wiki/List_comprehension
While notation in both math and CS varies, the components are always 
input source collection variables, conditions or predicates, and an 
output expression.

The Python reference manual documents comprehensions as an alternate 
atomic display form. In Chapter 6, Expressions, Section 2, Atoms,

"For constructing a list, a set or a dictionary Python provides special 
syntax called “displays”, each of them in two flavors:
either the container contents are listed explicitly, or
they are computed via a set of looping and filtering instructions, 
called a comprehension.
...
list_display ::=  "[" [expression_list | comprehension] "]"
<etc>"
A generator expression similarly represents an untyped abstract 
sequence, rather than a concrete class.
---

In summary: A context-manager, as an object with __enter__ and __exit__ 
methods, is not a proper component of a comprehension. For instance, 
replace "open('xxx')" in your proposal with a lock creation function. On 
the other hand, an iterable managed resource, as suggested by Mathias 
Panzenböck, works fine as a source. So it does work (as you noticed also).

-- 
Terry Jan Reedy