[Python-ideas] while conditional in list comprehension ??

[Zachary Ware]

On Jan 29, 2013 10:02 AM, "Oscar Benjamin" <oscar.j.benjamin at gmail.com>
wrote:
>
> On 29 January 2013 15:34, Zachary Ware <zachary.ware+pyideas at gmail.com>
wrote:
> >
> > On Jan 29, 2013 9:26 AM, "Oscar Benjamin" <oscar.j.benjamin at gmail.com>
> > wrote:
> >>
> >> On 29 January 2013 11:51, yoav glazner <yoavglazner at gmail.com> wrote:
> >> > Here is very similar version that works (tested on python27)
> >> >>>> def stop():
> >> > next(iter([]))
> >> >
> >> >>>> list((i if i<50 else stop()) for i in range(100))
> >> > [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,
19,
> >> > 20,
> >> > 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37,
38,
> >> > 39,
> >> > 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
> >>
> >> That's a great idea. You could also do:
> >> >>> list(i for i in range(100) if i<50 or stop())
> >>
> >> It's a shame it doesn't work for list/set/dict comprehensions, though.
> >>
> >
> > I know I'm showing my ignorance here, but how are list/dict/set
> > comprehensions and generator expressions implemented differently that
one's
> > for loop will catch a StopIteration and the others won't? Would it make
> > sense to reimplement list/dict/set comprehensions as an equivalent
generator
> > expression passed to the appropriate constructor, and thereby allow the
> > StopIteration trick to work for each of them as well?
>
> A for loop is like a while loop with a try/except handler for
> StopIteration. So the following are roughly equivalent:
>
> # For loop
> for x in iterable:
>     func1(x)
> else:
>     func2()
>
> # Equivalent loop
> it = iter(iterable)
> while True:
>     try:
>         x = next(it)
>     except StopIteration:
>         func2()
>         break
>     func1(x)
>
> A list comprehension is just like an implicit for loop with limited
> functionality so it looks like:
>
> # List comp
> results = [func1(x) for x in iterable if func2(x)]
>
> # Equivalent loop
> results = []
> it = iter(iterable)
> while True:
>     try:
>         x = next(it)
>     except StopIteration:
>         break
>     # This part is outside the try/except
>     if func2(x):
>         results.append(func1(x))
>
> The problem in the above is that we only catch StopIteration around
> the call to next(). So if either of func1 or func2 raises
> StopIteration the exception will propagate rather than terminate the
> loop. (This may mean that it terminates a for loop higher in the call
> stack - which can lead to confusing bugs - so it's important to always
> catch StopIteration anywhere it might get raised.)
>
> The difference with the list(generator) version is that func1() and
> func2() are both called inside the call to next() from the perspective
> of the list() function. This means that if they raise StopIteration
> then the try/except handler in the enclosing list function will catch
> it and terminate its loop.
>
> # list(generator)
> results = list(func1(x) for x in iterable if func2(c))
>
> # Equivalent loop:
> def list(iterable):
>     it = iter(iterable)
>     results = []
>     while True:
>         try:
>             # Now func1 and func2 are both called in next() here
>             x = next(it)
>         except StopIteration:
>             break
>         results.append(x)
>     return results
>
> results_gen = (func1(x) for x in iterable if func2(x))
> results = list(results_gen)
>

That makes a lot of sense. Thank you, Oscar and Joao, for the explanations.
I wasn't thinking in enough scopes :)

Regards,

Zach Ware
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130129/677790b0/attachment.html>