[Python-ideas] Membership of infinite iterators

[Steven D'Aprano]

On Tue, Oct 17, 2017 at 04:42:35PM +1000, Nick Coghlan wrote:

> I should also note that there's another option here beyond just returning
> "False": it would also be reasonable to raise an exception like
> "RuntimeError('Attempted negative containment check on infinite iterator')".

I don't think that works, even if we limit discussion to just 
itertools.count() rather than arbitrary iterators. Obviously we 
cannot wait until the entire infinite iterator is checked (that 
might take longer than is acceptible...) but if you only check a 
*finite* number before giving up, you lead to false-negatives:

# say we only check 100 values before raising
0 in itertools.count(1)  # correctly raises
101 in itertools.count(1)  # wrongly raises

If we do a computed membership test, then why raise at all? We quickly 
know whether or not the value is in the sequence, so there's no error to 
report.

Personally, I think a better approach is to give the specialist 
itertools iterator types a __contains__ method which unconditionally 
raises a warning regardless of whether the containment test returns 
true, false or doesn't return at all. Perhaps with a flag (module-wide?) 
to disable the warning, or turn it into an error.

I think a warning (by default) is better than an error because we don't 
really know for sure that it is an error:

n in itertools.count()

is, on the face of it, no more than an error than any other 
potentially infinite loop:

while condition(n):
    ...

and like any so-called infinite loop, we can never be sure when to give 
up and raise. A thousand loops? A million? A millisecond? An hour? 
Whatever we pick, it will be a case of one-size fits none.

I appreciate that, in practice it is easier to mess up a containment 
test using one of the itertools iterators than to accidentally write an 
infinite loop using while, and my concession to that is to raise a 
warning, and let the programmer decide whether to ignore it or turn it 
into an error.


-- 
Steve