<div dir="rtl"><div style dir="ltr">What is the "classic" use case for next() raising StopIteration, to be silently caught ? We need __next__ to do so in for loops, but when do we need it in the functional form?</div>
<div style dir="ltr"><br></div><div style dir="ltr">Elazar</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div dir="ltr">2014-02-21 11:25 GMT+02:00 Peter Otten <span dir="ltr"><<a href="mailto:__peter__@web.de" target="_blank">__peter__@web.de</a>></span>:</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">Oscar Benjamin wrote:<br>
<br>
> On 20 February 2014 22:38, אלעזר <<a href="mailto:elazarg@gmail.com">elazarg@gmail.com</a>> wrote:<br>
>><br>
>> 2014-02-21 0:00 GMT+02:00 Steven D'Aprano <<a href="mailto:steve@pearwood.info">steve@pearwood.info</a>>:<br>
>>><br>
>>> On Thu, Feb 20, 2014 at 04:14:17PM +0000, Oscar Benjamin wrote:<br>
>>> ><br>
>>> > The thing is just that bare next is not something that's widely<br>
>>> > recognised as being dangerous. I've seen examples of this kind of bug<br>
>>> > in samples from many Python aficionados (including at least one from<br>
>>> > you Terry).<br>
>>><br>
>>> Why is it dangerous, and what kind of bug?<br>
>>><br>
>>> If you're talking about the fact that next(it) can raise StopIteration,<br>
>>> I think you are exaggerating the danger. Firstly, quite often you don't<br>
>>> mind if it raises StopIteration, since that's what you would have done<br>
>>> anyway. Secondly, I don't see why raising StopIteration is so much more<br>
>>> dangerous than (say) IndexError or KeyError.<br>
>>><br>
>> I had this bug just the other day. I did not plan for the empty case,<br>
>> since it was obvious that the empty case is a bug, so I relied on the<br>
>> exception being raised in this case. But I did not get the exception<br>
>> since it was caught in a completely unrelated for loop. It took me a<br>
>> while to figure out what's going on, and it would've taken even more for<br>
>> someone else, not familiar with my assumption or with the whole<br>
>> StopIteration thing (which I believe is the common case). An IndexError<br>
>> or a KeyError would have been great in such a case.<br>
><br>
> Exactly. The bug I had manifested in a StopIteration that was raised<br>
> in a semi-deterministic (dependent on slowly varying data) fashion<br>
> after ~1 hour of processing. Had it resulted in an IndexError I would<br>
> have seen a traceback and could have fixed the bug within about 5<br>
> minutes.<br>
><br>
> But StopIteration doesn't necessarily bubble up in the same way as<br>
> other exceptions because it can be caught and silently supressed by a<br>
> for loop (or any consumer of iterators). So instead of a traceback I<br>
> had truncated output data. It took some time to discover and verify<br>
> that the output data was truncated. Then it took some time rerunning<br>
> the script under pdb which was no help since it couldn't latch into<br>
> the suppressed exception. I assumed that it must be an exception but<br>
> there were no try/except clauses anywhere in the code. Eventually I<br>
> found it by peppering the code with:<br>
><br>
> try:<br>
> ...<br>
> except Exception as e:<br>
> import pdb; pdb.set_trace()<br>
><br>
> It took most of a day for me to track that down instead of 5 minutes<br>
> precisely because StopIteration is not like other exceptions.<br>
> Admittedly I'd spot a similar bug much quicker now simply because I'm<br>
> aware of the possibility.<br>
><br>
> A simplified version of the bug is shown below:<br>
><br>
> def itermerge(datasources):<br>
> for source in datasources:<br>
> iterator = iter(source)<br>
> first = next(iterator)<br>
> for item in iterator:<br>
> yield first * item<br>
><br>
> data = [<br>
> [1, 1, 2, 3],<br>
> [1, 4, 5, 6],<br>
> [], # Who put that there?<br>
> [1, 7, 8, 9],<br>
> ]<br>
><br>
> for item in itermerge(data):<br>
> print(item)<br>
><br>
> If you run the above then you get:<br>
><br>
> $ python tmp.py<br>
> 1<br>
> 2<br>
> 3<br>
> 4<br>
> 5<br>
> 6<br>
><br>
> So the data is silently truncated at the empty iterable.<br>
><br>
>> It is *very* similar to the "and or" story.<br>
<br>
</div></div>I think the difference is that once you've learned the lesson you stop using<br>
`and...or` while you change your usage pattern for next() to minimal scopes<br>
<br>
def process_source(source):<br>
it = iter(source)<br>
first = next(it)<br>
for item in it:<br>
yield first * item<br>
<br>
def itermerge(sources):<br>
for source in sources:<br>
yield from process_source(source)<br>
<div class="HOEnZb"><div class="h5"><br>
<br>
_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" target="_blank">http://python.org/psf/codeofconduct/</a></div></div></blockquote></div><br></div>