[Python-ideas] Propagating StopIteration value

[Oscar Benjamin]

On 8 October 2012 03:40, Terry Reedy <tjreedy at udel.edu> wrote:
> On 10/7/2012 7:30 PM, Greg Ewing wrote:
>>
>> Oscar Benjamin wrote:
>>>
>>> Before pep 380 filter(lambda x: True, obj) returned an object that was
>>> the same kind of iterator as obj (it would yield the same values). Now
>>> the "kind of iterator" that obj is depends not only on the values that
>>> it yields but also on the value that it returns. Since filter does not
>>> pass on the same return value, filter(lambda x: True, obj) is no
>>> longer the same kind of iterator as obj.
>>
>>
>> Something like this has happened before, when the ability to
>> send() values into a generator was added. If you wrap a
>> generator with filter, you likewise don't get the same kind
>> of object -- you don't get the ability to send() things
>> into your filtered generator.
>>
>> So, "provide the same kind of iterator" is not currently part
>> of the contract of these functions.

They do provide the same kind of iterator in the sense that they
reproduce the properties of the object *in so far as it is an
iterator* by yielding the same values. I probably should have compared
filter(lambda x: True, obj) with iter(obj) rather than obj. In most
cases iter(obj) has a more limited interface.

send() is clearly specific to generators: user defined iterator
classes can provide any number of state-changing methods (usually with
more relevant names) but this is difficult for generators so a generic
mechanism is needed. The return value attached to StopIteration
"feels" more fundamental to me since there is now specific language
syntax both for extracting it and for returning it in generator
functions.

> Iterators are Python's generic sequential access device. They do that one
> thing and do it well.
>
> The iterator protocol is intentionally and properly minimal. An iterator
> class *must* have appropriate .__iter__ and .__next__ methods. It *may* also
> have any other method and any data attribute. Indeed, any iterator much have
> some specific internal data. But these are ignored in generic iterator (or
> iterable) functions. If one does not want that, one should write more
> specific code.

Generalising the concept of an iterator this way is entirely backwards
compatible with existing iterators and does not place any additional
burden on defining iterators: most iterators can simply be iterators
that return None. The feature is optional for any iterator but this
thread is about whether it should be optional for a generic processor
of iterators.

> Serhily, if you want a module of *generator* specific functions ('gentools'
> ?), you should write one and submit it to pypi for testing.

This is probably the right idea. As the feature gains use cases the
best way to handle it will become clearer.


Oscar