[Tutor] Python Generator expressions

[Steven D'Aprano]

On Tue, Jul 23, 2019 at 10:36:01PM +0530, Animesh Bhadra wrote:
> Hi All,
> 
> Need one help in understanding generator expression/comprehensions.
> 
> This is my sample code.

Lots of missing spaces in your code! Don't forget to hit the space bar 
between words :-)

Also missing indentation, which is probably Gmail being "helpful".

I'm just going to fix the obvious typos without comment.


> # This code creates a generator and not a tuple comprehensions.
> my_square =(num*num for num in range(11))
> print(my_square) # <generator object <genexpr> at 0x7f3c838c0ca8>
> # We can iterate over the square generator like this.
> try:
>     while True:
>         print(next(my_square)) # Prints the value 0,1,4....
> except StopIteration as SI:
>     print("Stop Iteration")
> # Another iteration
> for x in my_square:
>     print(x) # This prints nothing.


> Does the generator exhausts its values when we run the iterator once?

Correct.


> Lastly any specific reason for not having a tuple comprehensions?

(1) History and (2) because tuple comprehensions are not useful enough 
to dedicate syntax to them.


Originally, Python introduced *list comprehensions* using square bracket 
syntax:

    [expression for x in values]

which created a list. If I remember correctly, I think this was about 
Python 2.2. 

Tuples are mostly intended for *small* numbers of heterogeneous values, 
like a struct or record in other languages:

    # Typical example of tuple
    (1, "Hello", 4.5)

while lists are intended for potentially large numbers of homogeneous 
values, like an array of data:

    [2.3, 4.5, 0.9, 7.2, 9.3, 4.5, 6.1, 0.2, 8.7, ... # and many more

(heterogeneous = "different types", homogeneous = "same types")

That makes lists a really good fit for comprehensions, and tuples a bad 
fit. So it was decided that the need for a direct tuple comprehension 
was pretty low. If you wanted a tuple, you could easily get one:

    tuple([expression for x in values])

and that was good enough without wasting useful syntax on something that 
would hardly ever be needed.

It would be a tiny bit more expensive to run (first you create a list, 
then copy it to a tuple, and then garbage collect the list) but so what? 
Not every rare operation needs to be optimized.

A few years later, generator comprehensions were added as a lazy version 
of list comprehensions. List comprehensions produce all the values up 
front, while generator comprehensions produce them one at a time as 
needed. The syntax chosen used round brackets ( ... ) instead of square 
brackets [ ... ] but otherwise was identical.

What other syntax could have been chosen? Curly brackets are used for 
dicts (Python 2 and 3) and sets (Python 3 only), so they are unsuitable, 
and later on in Python 3 we gained dict and set comprehensions as well.

Had we wasted the ( ... ) syntax on tuples, which hardly anyone would 
have ever used, we would have missed out on the super-useful lazy 
generator comprehensions.


 
> Have checked this link, but could not understood the reason?
> 
>  * 
>  https://stackoverflow.com/questions/16940293/why-is-there-no-tuple-comprehension-in-python

Don't believe everything you read on Stackoverflow. There's lots of 
utter nonsense on that page, for example:

    "The answer is obviously because tuple syntax and parenthesis are 
    ambiguous"

is rubbish. The parser can tell the difference between tuples and 
parentheses. Tuples use *commas*, not parentheses, except for the 
empty tuple () which is a special case. There's no ambiguity in Python.

The top rated answer

    "parentheses were already taken for … generator expressions"

is incorrect, because generator expressions came second, not first. 
Python could have added tuple comprehensions at the same time it added 
list comprehensions, but the developers decided not to:

- it wasn't clear how useful comprehensions would be; they were really 
useful in Haskell, but people weren't sure if Python developers would 
take to them (they sure did!);

- list comprehensions are clearly useful, but tuple comprehensions not 
so much. Not ever single use has to get its own special syntax.

Of course, *later on* once Python had generator comprehensions using 
parentheses, we couldn't easily add tuple comprehensions because 
there's no good syntax left. But why would we want them?



-- 
Steven