On 18 October 2017 at 20:39, Koos Zevenhoven
On Oct 18, 2017 13:29, "Nick Coghlan"
wrote: On 18 October 2017 at 19:56, Koos Zevenhoven
wrote: I'm unable to reproduce the "uninterruptible with Ctrl-C" problem with infinite iterators. At least itertools doesn't seem to have it:
import itertools for i in itertools.count(): ... pass ...
That's interrupting the for loop, not the iterator. This is the test case you want for the problem Jason raised:
>>> "a" in itertools.count()
Be prepared to suspend and terminate the affected process, because Ctrl-C isn't going to help :)
I'm writing from my phone now, cause I was dumb enough to try list(count())
Yeah, that's pretty much the worst case example, since the machine starts thrashing memory long before it actually gives up and starts denying the allocation requests :(
But should it be fixed in list or in count?
That one can only be fixed in count() - list already checks operator.length_hint(), so implementing itertools.count.__length_hint__() to always raise an exception would be enough to handle the container constructor case. The open question would then be the cases that don't pre-allocate memory, but still always attempt to consume the entire iterator: min(itr) max(itr) sum(itr) functools.reduce(op, itr) "".join(itr) And those which *may* attempt to consume the entire iterator, but won't necessarily do so: x in itr any(itr) all(itr) The items in the first category could likely be updated to check length_hint and propagate any errors immediately, since they don't provide any short circuiting behaviour - feeding them an infinite iterator is a guaranteed uninterruptible infinite loop, so checking for a length hint won't break any currently working code (operator.length_hint defaults to returning zero if a type doesn't implement __length_hint__). I'm tempted to say the same for the APIs in the latter category as well, but their short-circuiting semantics mean those can technically have well-defined behaviour, even when given an infinite iterator: >>> any(itertools.count()) True >>> all(itertools.count()) False >>> 1 in itertools.count() True It's only the "never short-circuits" branch that is ill-defined for non-terminating input. So for these, the safer path would be to emit DeprecationWarning if length_hint fails in 3.7, and then pass the exception through in 3.8+. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia