[Python-ideas] Introduce collections.Reiterable

[Antoine Pitrou]

Le Thu, 19 Sep 2013 21:28:04 -0400,
Terry Reedy <tjreedy at udel.edu> a écrit :
> The first problem, which impinges on both specification and
> reiteration, is than an iterable may be either finite, or not, or 'in
> between' depending the hardware and user needs.

This isn't a problem. itertools.tee() will deal with it fine.

> An additional complication, including for reiteration, is that 
> 'practically' finite may be different for time and space.

This is a strawman, since the "complication" applies to all kinds of
iterables.

> Currently, if dev needs to iterate an input more than once, the 
> specification should say so. If the user wants to pass an iterator,
> the user can instead pass list(iter).

Not if the user really wants, or needs, the iterator to be consumed
lazily. This can matter if the iterator is infinite, or if consuming it
has resouce-consuming side effects such as doing I/O, etc.

list(iter) is a limited solution to the problem. And the thing is,
using a Reiterable helper doesn't preclude the caller from calling
list() as well, so it's a strawman here.

> Now to the varieties of reiteration:
> 
> A. Serial: [...]
> 
> B. Parallel: [...]
> 
> C: Crossed: [...]

Nice discussion, but unrelated. If the iterable doesn't work in
those situations, it is purely a bug in the iterable, and it's not
related to "reiteration".

In other words, if an API returns something that cannot be iterated
an arbitrary number of times, it should return an iterator, not an
iterable ;-)

> However, another hidden assumption in this thread has been that 
> non-iterator iterables are deterministic, in the sense that
> re-calling iter(it) returns an iterator that yields the same sequence
> of items before raising StopIteration. Some very useful
> iterator-producing functions do not do that (ones returning iterators
> based on pseudo-random or external inputs).

Well, I hardly ever use non-deterministic iterables, and I can't
remember passing a "pseudo-random iterator" to a function expecting a
generic iterable. YMMV.

> So we need to add
> 'deterministic' to the notion of 'reiterable'. And that cannot be
> mechanically determined.

Many things are not mechanically determined that still make sense to
specifiy in an API. "Mechanically determined" is a rather silly
criterion when designing APIs, especially in a dynamic language where
nothing can ever be taken for granted.

(in other words, if you want "mechanically determined" API guarantees,
perhaps you should try Haskell or Rust :-))

> (Other possible complications: a resource can only be accessed by one 
> connection at a time. Or it limits the frequency of connections.)

That's true, but the caller can still call list() regardless of how the
callee is implemented.

Regards

Antoine.