[Python-Dev] generator expression syntax

[Gareth McCaughan]

On Wednesday 2004-03-24 05:46, Robert Mollitor wrote:

> The idea of generator expressions seems good, but the proposed syntax
> seems a little wrong to me.
> 
> First, the syntax is too dependent on the parentheses.  To my mind, 
> this is a fourth meaning for parentheses.  (1) Parentheses group and
> associate expressions: "(a * (b + c))", "if (a and b)".  (2) Parentheses
> construct tuples: "(1, 2, 3)", "()", "('a',)". (3) Parentheses enclose
> argument lists (arguably a special case of tuple-constructor):
> "def f(a, b, c)", "obj.dump()", "class C (A, B)".  And now (4*)
> generator expressions: "(x*x for x in list)".  I realize that in some
> sense the parentheses are not part of the expression syntax (since we
> wouldn't require "sum((x * x for x in list))"), but they are mandatory
> nonetheless because you can't have "a = x*x for x in list".  This seems
> like it stretching a familiar construct too far.

Actually 4 and 2 are very similar. It's *commas* that construct tuples,
not parentheses; the parentheses are just necessary in some contexts
to shield the tuple construction from other things going on nearby.
Requiring parens around a generator expression is just like requiring
parens around a single-element tuple expression.

> Second, it looks like a "tuple comprehension".  The list comprehension
> construct yields a list.  A generator expression looks like it should yield
> a tuple, but it doesn't.

OK, so generator expressions should be wrapped in the same punctuation
that's used to write explicit generators. Er, except that there isn't
any.

> Third, it seems Lisp-ish or Objective-C-ish and not Pythonic to me.
> I realize that is just a style thing, but that's the flavor I get.

There are two separate statements here: (1) it seems Lisp-ish or
ObjC-ish, and (2) it doesn't seem Pythonic. #1 isn't a problem
in itself; are you deducing #2 from #1 or claiming that it's true
in its own right? (I don't see either #1 or #2 myself, but I find
#1 easier to sympathize with than #2.)

> Fourth, it seems like this variable binding thing will trip people up
> because it is not obvious that a function is being defined.  Lambdas
> have variable binding issues, but that is obviously a special construct.

It's not clear to me that lambdas are any more obviously a special
construct than generator expressions. Are you sure it's not just
that you're used to lambdas and know that they're special, but aren't
yet used to generator expressions?

> OK, I not completely sure if this will work to everyone's satisfaction, 
> but here is my proposal:  replace
> the
> 
> 	(x*x for x in a)
> 
> construct with
> 
> 	lambda: yield x*x for x in a

I predict that to a good first approximation this one won't work
to *anyone's* satisfaction.

> 
> CONS
> 
>   - "Ick, lambdas"
>   -  It's longer

  - lambda doesn't mean "generator expression" or "thing involving
    variable binding" or anything like it; it means "anonymous
    function"
  - it introduces a magical new thing that you can do *only* in
    lambdas and not in named functions
  - The feature of ordinary generators that this trades on (namely
    that a generator definition makes something that you call in
    order to get an iterable thing) is probably the single most
    obscure point about generators
  - the variable binding features that lambdas already have aren't
    the same ones that are being proposed for generator expressions
  - it's as ugly as, ummm, something very ugly

> PROS
> 
>   - Lambdas are funky, but they are explicitly funky: look up 'lambda' 
> in the index and go to that section of the book

This "pro" would equally justify overloading "if" or inventing
a keyword "wombiquangle" for this purpose.

>   - Use the variable binding rules of lambas and people will be as happy 
> with that as they are with lambdas in the first place (for better or worse)

The use cases of lambdas and of generator expressions are not
the same. Why should the appropriate variable binding rules
be the same? And why, therefore, should people be equally
happy with them?

> So here would be the recasting of some of examples in PEP 289:
> 
> 	sum(lambda: yield x*x for x in range(10))
> 
> 	d = dict (lambda: yield x, func(k) for k in keylist)
[etc]

I can't believe you typed all those in and looked at them and
still like this idea :-).

> 	# I think the following would work, too
> 	for xx in lambda: yield x*x for x in range(10):
> 		print xx
> 
> 	# If so, how's this for decadent
> 	for i,a in lambda: yield i,list[i] for i in range(len(list)):
> 		print i, a

You think this is a *good* thing? Compare these, with their
misleading initial segments "for something in lambda:" and all,
with

    for xx in (x*x for x in range(10)):
        print x

    for i,a in ((i,list[i]) for i in range(len(list))):
        print i,a

which, for all their allegedly superfluous parentheses
and allegedly weird variable binding behaviour, I can
at least read.

-- 
g