<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 18 October 2017 at 20:39, Koos Zevenhoven <span dir="ltr"><<a href="mailto:k7hoven@gmail.com" target="_blank">k7hoven@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div><span class="gmail-"><div dir="auto" style="font-family:sans-serif;font-size:13.696px"><span style="font-size:large">On Oct 18, 2017 13:29, "Nick Coghlan" <<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>> wrote:</span><br></div></span><span class="gmail-"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail-m_-2423510541164657263quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="gmail-m_-2423510541164657263quoted-text">On 18 October 2017 at 19:56, Koos Zevenhoven <span dir="ltr"><<a href="mailto:k7hoven@gmail.com" target="_blank">k7hoven@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span class="gmail-m_-2423510541164657263m_183500653703616643gmail-"></span><div class="gmail_extra"><div class="gmail_quote"><span class="gmail-m_-2423510541164657263m_183500653703616643gmail-"></span><div style="font-family:monospace,monospace">I'm unable to reproduce the "uninterruptible with Ctrl-C" problem with infinite iterators. At least itertools doesn't seem to have it:</div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace"><div>>>> import itertools</div><div>>>> for i in itertools.count():</div><div>... pass</div><div>... </div></div></div></div></div></blockquote><div><br></div></div><div>That's interrupting the for loop, not the iterator. This is the test case you want for the problem Jason raised:</div><div><br></div><div> >>> "a" in itertools.count()</div><div><br></div><div>Be prepared to suspend and terminate the affected process, because Ctrl-C isn't going to help :)<br></div></div></div></div></blockquote></div></div></span></div><div dir="auto"><br></div><div dir="auto"><span class="gmail-"><div dir="auto" style="font-family:sans-serif;font-size:13.696px">I'm writing from my phone now, cause I was dumb enough to try list(count())</div></span></div></div></blockquote><div><br></div><div>Yeah, that's pretty much the worst case example, since the machine starts thrashing memory long before it actually gives up and starts denying the allocation requests :(<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="auto"><span class="gmail-"><div dir="auto" style="font-family:sans-serif;font-size:13.696px">But should it be fixed in list or in count?</div></span></div></div></blockquote><div><br></div><div>That one can only be fixed in count() - list already checks operator.length_hint(), so implementing itertools.count.__length_hint__() to always raise an exception would be enough to handle the container constructor case.<br></div></div><div class="gmail_quote"><br></div><div class="gmail_quote">The open question would then be the cases that don't pre-allocate memory, but still always attempt to consume the entire iterator:</div><div class="gmail_quote"><br></div> min(itr)<br><div class="gmail_quote"> max(itr)</div><div class="gmail_quote"> sum(itr)<br></div><div class="gmail_quote"> functools.reduce(op, itr)</div><div class="gmail_quote"> "".join(itr)<br></div><div class="gmail_quote"><br></div><div class="gmail_quote">And those which *may* attempt to consume the entire iterator, but won't necessarily do so:</div><div class="gmail_quote"><br></div><div class="gmail_quote"> x in itr</div><div class="gmail_quote"> any(itr)</div><div class="gmail_quote"> all(itr)</div><div class="gmail_quote"><br></div><div class="gmail_quote">The items in the first category could likely be updated to check length_hint and propagate any errors immediately, since they don't provide any short circuiting behaviour - feeding them an infinite iterator is a guaranteed uninterruptible infinite loop, so checking for a length hint won't break any currently working code (operator.length_hint defaults to returning zero if a type doesn't implement __length_hint__).</div><div class="gmail_quote"><br></div><div class="gmail_quote">I'm tempted to say the same for the APIs in the latter category as well, but their short-circuiting semantics mean those can technically have well-defined behaviour, even when given an infinite iterator:</div><div class="gmail_quote"><br></div><div class="gmail_quote"> >>> any(itertools.count())<br> True<br> >>> all(itertools.count())<br> False<br> >>> 1 in itertools.count()<br> True</div><div class="gmail_quote"><br></div><div class="gmail_quote">It's only the "never short-circuits" branch that is ill-defined for non-terminating input. So for these, the safer path would be to emit DeprecationWarning if length_hint fails in 3.7, and then pass the exception through in 3.8+.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Cheers,</div><div class="gmail_quote">Nick.<br></div><div class="gmail_quote"></div><div class="gmail_quote"><br></div>-- <br><div class="gmail_signature">Nick Coghlan | <a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a> | Brisbane, Australia</div>
</div></div>