When use a generator from different threads you can get a ValueError "generator already executing". Getting this exception with the single thread is a programming error, it in case of different threads it could be possible to wait until other thread finish executing the generator. The generator can be made thread-safe after wrapping it in a class that acquire a lock before calling the generator's __next__ method (for example see [1]). But this is not very efficient of course. I wondering if it is worth to add support of thread-safe generators in the stdlib. Either by providing standard decorator (written in C for efficiency), or adding threading support just in the generator object. The latter may need increasing the size of the generator object for a lock and thread identifier (but may be GIL is enough), but should not affect performance since locking is used only when you faced with a generator running in other thread. This topic already was raised on Python-Dev [2] but didn't moved too much. There are a number of StackOverflow questions about threads and generators. We have already encountered this issue in the stdlib. Once in regrtest with the -j option ([3], [4]), other time after reimplementing tempfile._RandomNameSequence as a generator [5]. [1] http://anandology.com/blog/using-iterators-and-generators/ [2] https://mail.python.org/pipermail/python-dev/2004-February/042390.html [3] https://bugs.python.org/issue7996 [4] https://bugs.python.org/issue15320 [5] https://bugs.python.org/issue30030
Serhiy Storchaka wrote:
I don't think that's true, because the first thread to use a generator has to lock it as well. And even if there is only one thread in existence when __next__ is called, another one could be created before it finishes and try to use the generator. -- Greg
On 15.04.17 01:42, Greg Ewing wrote:
The first thread just sets the running flag (as in current code). Due to GIL this doesn't need additional synchronization. Other threads check this flag and sleep rather than raising an exception. After finishing the generator the first thread checks if there are other threads awaiting and awake a one of them.
On 15 April 2017 at 02:47, Serhiy Storchaka <storchaka@gmail.com> wrote:
Allowing multiple worker threads to pull from the same work queue is a general concurrency problem, and that's why we have queue.Queue in the standard library: https://docs.python.org/3/library/queue.html So I'd be opposed to trying to make generator objects natively thread aware - as Stephen notes, the GIL is an implementation detail of CPython, so it isn't OK to rely on it when defining changes to language level semantics (in this case, whether or not it's OK to have multiple threads all calling the same generator without some form of external locking). However, it may make sense to explore possible options for offering a queue.AutoQueue type, where the queue always has a defined maximum size (defaulting to 1), disallows explicit calls to put(), and automatically populates itself based on an iterator supplied to the constructors. Once the input iterator raises StopIteration, then the queue will start reporting itself as being empty. The benefit of going down that path is that it can be used with arbitrary iterators (not just generators), and can be more easily generalised to other synchronisation models (such as multiprocessing). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
+1 A generator that can have values pulled from it on different threads sounds like a queue to me, so the AutoQueue class that wraps a generator seems like a natural abstraction to work with. It also means that the cost for thread safety is only paid by those applications that need it. Paul
On 17 April 2017 at 08:00, Paul Moore <p.f.moore@gmail.com> wrote:
If someone did build something like this, it would be interesting to benchmark it against a more traditional producer thread model, where one thread is responsible for adding work items to the queue, while others are responsible for draining them. The trick is that an auto-queue would borrow execution time from the consumer threads when new values are needed, so you'd theoretically get fewer context switches between threads, but at the cost of changing the nature of the workload in a given thread, and hence messing with the working set of objects it has active. It may also pair well with the concurrent.futures.Executor model, which is already good for "go handle this predefined list of tasks", but currently less useful as a replacement for a message queue with a pool of workers. Setting the latter up yourself is currently still a bit tedious, since: 1. we don't have a standard threading Pool abstraction in the standard library, just the one tucked away as part of multiprocessing 2. while queue.Queue has native support for worker pools, we don't provide a pre-assembled version that makes it easy to say "here is the producer, here are the consumers, wire them together for me" There are good reasons for that (mainly that it's hard to come up with an abstraction that's useful in its own right without becoming so complex that you're on the verge of reinventing a task manager like celery or a distributed computation manager like dask), but at the same time, the notion of "input queue, worker pool, output queue" is one that comes up a *lot* across different concurrency models, so there's potential value in providing a low-barrier-to-entry introduction to that idiom as part of the standard library. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
This is a bad idea in the generator itself, as commented earlier by others here.
But any reliance on gi_running, as seen here, can only be considered to be possible help in detecting such races; it would not even come close to preventing a race: https://github.com/jythontools/jython/blob/master/src/org/python/core/PyGene... (We don't even bother making gi_running a volatile to get actual test-and-set style semantics, because really it makes no sense to pretend otherwise; and why pay the performance penalty?) The idea of putting generators behind a queue sounds reasonably workable - the semantics then are the right ones, although implementing this efficiently is the trick here. On Sun, Apr 16, 2017 at 11:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Thread safety is very complex and has an impact on performance. I dislike the idea of providing such property to generators which can have a complex next method. IMHO it's better to put a generator in wrapper which adds thread safety. What do you think? Victor Le 14 avr. 2017 18:48, "Serhiy Storchaka" <storchaka@gmail.com> a écrit :
Serhiy Storchaka wrote:
I don't think that's true, because the first thread to use a generator has to lock it as well. And even if there is only one thread in existence when __next__ is called, another one could be created before it finishes and try to use the generator. -- Greg
On 15.04.17 01:42, Greg Ewing wrote:
The first thread just sets the running flag (as in current code). Due to GIL this doesn't need additional synchronization. Other threads check this flag and sleep rather than raising an exception. After finishing the generator the first thread checks if there are other threads awaiting and awake a one of them.
On 15 April 2017 at 02:47, Serhiy Storchaka <storchaka@gmail.com> wrote:
Allowing multiple worker threads to pull from the same work queue is a general concurrency problem, and that's why we have queue.Queue in the standard library: https://docs.python.org/3/library/queue.html So I'd be opposed to trying to make generator objects natively thread aware - as Stephen notes, the GIL is an implementation detail of CPython, so it isn't OK to rely on it when defining changes to language level semantics (in this case, whether or not it's OK to have multiple threads all calling the same generator without some form of external locking). However, it may make sense to explore possible options for offering a queue.AutoQueue type, where the queue always has a defined maximum size (defaulting to 1), disallows explicit calls to put(), and automatically populates itself based on an iterator supplied to the constructors. Once the input iterator raises StopIteration, then the queue will start reporting itself as being empty. The benefit of going down that path is that it can be used with arbitrary iterators (not just generators), and can be more easily generalised to other synchronisation models (such as multiprocessing). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 15 April 2017 at 10:45, Nick Coghlan <ncoghlan@gmail.com> wrote:
+1 A generator that can have values pulled from it on different threads sounds like a queue to me, so the AutoQueue class that wraps a generator seems like a natural abstraction to work with. It also means that the cost for thread safety is only paid by those applications that need it. Paul
On 17 April 2017 at 08:00, Paul Moore <p.f.moore@gmail.com> wrote:
If someone did build something like this, it would be interesting to benchmark it against a more traditional producer thread model, where one thread is responsible for adding work items to the queue, while others are responsible for draining them. The trick is that an auto-queue would borrow execution time from the consumer threads when new values are needed, so you'd theoretically get fewer context switches between threads, but at the cost of changing the nature of the workload in a given thread, and hence messing with the working set of objects it has active. It may also pair well with the concurrent.futures.Executor model, which is already good for "go handle this predefined list of tasks", but currently less useful as a replacement for a message queue with a pool of workers. Setting the latter up yourself is currently still a bit tedious, since: 1. we don't have a standard threading Pool abstraction in the standard library, just the one tucked away as part of multiprocessing 2. while queue.Queue has native support for worker pools, we don't provide a pre-assembled version that makes it easy to say "here is the producer, here are the consumers, wire them together for me" There are good reasons for that (mainly that it's hard to come up with an abstraction that's useful in its own right without becoming so complex that you're on the verge of reinventing a task manager like celery or a distributed computation manager like dask), but at the same time, the notion of "input queue, worker pool, output queue" is one that comes up a *lot* across different concurrency models, so there's potential value in providing a low-barrier-to-entry introduction to that idiom as part of the standard library. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
This is a bad idea in the generator itself, as commented earlier by others here.
But any reliance on gi_running, as seen here, can only be considered to be possible help in detecting such races; it would not even come close to preventing a race: https://github.com/jythontools/jython/blob/master/src/org/python/core/PyGene... (We don't even bother making gi_running a volatile to get actual test-and-set style semantics, because really it makes no sense to pretend otherwise; and why pay the performance penalty?) The idea of putting generators behind a queue sounds reasonably workable - the semantics then are the right ones, although implementing this efficiently is the trick here. On Sun, Apr 16, 2017 at 11:08 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
Thread safety is very complex and has an impact on performance. I dislike the idea of providing such property to generators which can have a complex next method. IMHO it's better to put a generator in wrapper which adds thread safety. What do you think? Victor Le 14 avr. 2017 18:48, "Serhiy Storchaka" <storchaka@gmail.com> a écrit :
participants (8)
-
Greg Ewing -
Guido van Rossum -
Jim Baker -
Nick Coghlan -
Paul Moore -
Serhiy Storchaka -
Stephen J. Turnbull -
Victor Stinner