[Python-Dev] semantics of subclassing things from itertools

[Terry Reedy]

On 9/10/2015 3:23 AM, Maciej Fijalkowski wrote:
> Hi
>
> I would like to know what are the semantics if you subclass something
> from itertools (e.g. islice).

I believe people are depending on an undocumented internal speed 
optimization.  See below.

> Right now it's allowed and people do it, which is why the
> documentation is incorrect. It states "equivalent to: a function-or a
> generator",

I believe Raymond has said that 'equivalent' should be taken as 
'equivalent in essential function for iterating' rather than 'exactly 
equivalent for all operations'.  The results of type() and isinstance() 
are not relevant for this.  The itertools doc begins with "This module 
implements a number of iterator building blocks ..." After listing them, 
it says "The following module *functions* all construct and return 
iterators." (These part of the doc are unchanged from the original in 
2.3. I added the emphasis.) The generator functions are mathematically 
equivalent if they produce equivalent iterators for the same inputs. 
The iterators are equivalent if they produce the same stream of objects 
when iterated. If they do, the doc is correct; if not, the doc is buggy 
and should be fixed.

I see the undocumented fact that the module *functions* are implemented 
as C classes as an internal implementation detail to optimize speed.  I 
believe Raymond intentionally used 'function' rather than 'class' and 
intended the equivalents to be usable by other implementations. Ask 
Raymond directly (he is not currently active on pydev) if I am correct.

> but you can't subclass whatever it is equivalent to, which
> is why in PyPy we're unable to make it work in pure python.

You could write equivalent iterator classes in Python, but the result 
would be significantly slower.

-- 
Terry Jan Reedy