[Python-ideas] Thread-safe generators

[Paul Moore]

On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan at gmail.com> wrote:
> So I'd be opposed to trying to make generator objects natively thread
> aware - as Stephen notes, the GIL is an implementation detail of
> CPython, so it isn't OK to rely on it when defining changes to
> language level semantics (in this case, whether or not it's OK to have
> multiple threads all calling the same generator without some form of
> external locking).
>
> However, it may make sense to explore possible options for offering a
> queue.AutoQueue type, where the queue always has a defined maximum
> size (defaulting to 1), disallows explicit calls to put(), and
> automatically populates itself based on an iterator supplied to the
> constructors. Once the input iterator raises StopIteration, then the
> queue will start reporting itself as being empty.

+1 A generator that can have values pulled from it on different
threads sounds like a queue to me, so the AutoQueue class that wraps a
generator seems like a natural abstraction to work with. It also means
that the cost for thread safety is only paid by those applications
that need it.

Paul