[Python-ideas] With clauses for generator expressions

Terry Reedy tjreedy at udel.edu
Sat Nov 17 01:00:19 CET 2012

```On 11/16/2012 4:09 AM, Andrew Barnert wrote:
> So far, nearly everyone is discussing things which are tangential, or arguing
> that one of the optional variants is bad. So let me strip down the proposal,
> without any options in it, and expand on a use case. The syntax is:
>
>
>      (foo(line) with open('bar') as f for line in baz(f))

OK, that's helpful. Now let me strip down my objection to this: your
proposal is conceptually wrong because it mixes two distinct and
different ideas -- collection definition and context management. It
conflicts with a well-defined notion of long standing.

To explain: in math, one can define a set explicitly by displaying the
members or implicitly as a subset of based on one or more base sets.
Using one version of the notation
{0, 2, 4} == {2*i| i in N; i < 3}
The latter is 'set-builder notation' or a 'set comprehension' (and would
usually use the epsilon-like member symbol instead of 'in'). The idea
goes back at least a century.
https://en.wikipedia.org/wiki/Set-builder_notation

In Python, the latter directly translates to
{2*i for i in itertools.count() if i < 3} ==
{i for i in range(0, 5, 2)}
(Python does not require the base collection to match the result class.)
Another pair of examples:
{(i,j)| i in N, j in N; i+j <= 5}
{(i,j) for i in count() for j in count if i+j <= 5}

Similar usage in programming go back over half a century.
https://en.wikipedia.org/wiki/List_comprehension
While notation in both math and CS varies, the components are always
input source collection variables, conditions or predicates, and an
output expression.

The Python reference manual documents comprehensions as an alternate
atomic display form. In Chapter 6, Expressions, Section 2, Atoms,

"For constructing a list, a set or a dictionary Python provides special
syntax called “displays”, each of them in two flavors:
either the container contents are listed explicitly, or
they are computed via a set of looping and filtering instructions,
called a comprehension.
...
list_display ::=  "[" [expression_list | comprehension] "]"
<etc>"
A generator expression similarly represents an untyped abstract
sequence, rather than a concrete class.
---

In summary: A context-manager, as an object with __enter__ and __exit__
methods, is not a proper component of a comprehension. For instance,
replace "open('xxx')" in your proposal with a lock creation function. On
the other hand, an iterable managed resource, as suggested by Mathias
Panzenböck, works fine as a source. So it does work (as you noticed also).

--
Terry Jan Reedy

```