[Python-ideas] Add closing and iteration to threading.Queue

[Cameron Simpson]

On 21Oct2018 18:06, Nathaniel Smith <njs at pobox.com> wrote:
>On Sun, Oct 21, 2018, 16:48 MRAB <python at mrabarnett.plus.com> wrote:
>> On 2018-10-21 22:30, Antoine Pitrou wrote:
>> > Ah.  This is the one statement that makes me favorable to this 
>> > idea.
>> > When there is a single consumer, it's easy enough to send a sentinel.
>> > But when there are multiple consumers, suddenly you must send exactly
>> > the right number of sentinels (which means you also have to careful
>> > keep track of their number, which isn't always easy).  There's some
>> > delicate code doing exactly that in concurrent.futures.
>> >
>> You don't need more than one sentinel. When a consumer sees the
>> sentinel, it just needs to put it back for the other consumers.

Yes, this is exactly what my own IterableQUeue does.

>I'm not sure if this is an issue the way Queue is used in practice, but in
>general you have to be careful with this kind of circular flow because if
>your queue communicates backpressure (which it should) then circular flows
>can deadlock.

Haven't come across this myself. A closeable queue doesn't seem circular 
to me. The handling of the sentinel is internal to the IterableQueue, so 
external users never see it.

Cheers,
Cameron Simpson <cs at cskk.id.au>