[Python-ideas] Simpler thread synchronization using "Sticky Condition"

[Nathaniel Smith]

On Tue, Mar 26, 2019, 09:50 Richard Whitehead <richard.whitehead at ieee.org>
wrote:

> Nathaniel,
>
> Thanks very much for taking the time to comment.
>
> Clearing the event after waiting for it will introduce a race condition:
> if
> the sender has gone around its loop again and set the semaphore after we
> have woken but before we've cleared it.


Sounds fine to me. Why is that a problem? Can you write down an example of
how two threads could be interleaved to produce incorrect results?

As you said, this stuff is tricky!
> The only safe way is to make the wait-and-clear atomic, which can be done
> with a lock; and this comes essentially back to what I'm proposing.
>
> I realise this is not a fundamental new primitive - if it was, I wouldn't
> be
> able to build it in pure Python - but I've found it extremely useful in
> our
> generic threading and processing library.
>
> You're right about what you say regarding queues; I didn't want to go into
> the full details of the multi-threading and multi-processing situation at
> hand, but I will say that we have a pipeline of tasks that can run as
> either
> threads or processes, and we want to make it easy to construct this
> pipeline, "wiring" it as necessary; combining command queues with data
> queues just gets a real mess.
>

But you're effectively implementing a multi-producer single-consumer Queue
anyway, so without any details it's hard to guess why using a Queue would
be messier. I know you don't want to get into too many details, but if the
whole motivation for your proposal is based on some details then it's
usually a good idea to explain them :-).

-n

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20190326/a714ad7a/attachment-0001.html>