[Python-ideas] make Connections iterable

[Antoine Pitrou]

On Tue, 9 Jan 2018 20:46:35 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 9 January 2018 at 20:07, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Mon, 8 Jan 2018 21:22:56 -0800
> > Nathaniel Smith <njs at pobox.com> wrote:  
> >> I'm surprised that multiprocessing.Connection isn't iterable -- it seems
> >> like an obvious oversight.  
> >
> > What is obvious about making a connection iterable?  It's the first
> > time I see someone requesting this.  
> 
> If you view them as comparable to subprocess pipes, then it can be
> surprising that they're not iterable when using a line-oriented
> protocol.
> 
> If you instead view them as comparable to socket connections, then the
> lack of iteration support seems equally reasonable.

multiprocessing connections are actually message-oriented.  So perhaps
it could make sense for them to be iterable.  But they are also quite
low-level (often you wouldn't use them directly, but instead rely on
multiprocessing.Queue).

> Hence my suggestion of providing a docs recipe showing an example of
> wrapping a connection in a generator in order to define a suitable way
> of getting from a raw bytestream to iterable chunks.

Well... if someone needs a doc recipe for this, they shouldn't use the
lower-level functionality and instead stick to multiprocessing.Queue.

(this begs the question: should multiprocessing.Queue be iterable?
well, it's modeled on queue.Queue which isn't iterable)

Regards

Antoine.