It does make sense to have a barrier synchronization primitive for asyncio. The idea is to make a coroutine block until at least X coroutines are waiting to enter the barrier. This is very useful, if certain actions need to be synchronized. Recently, I had to implement a barier myself for our use case. See code below: It is simple to implement, but I too would like to have one for asyncio, in order to be consistent with the concurrency primitives we have for threading. Jonathan class Barier: """ Make a coroutine block until there are at least X waiters. Similar to the threading Barier objects but for asyncio: https://docs.python.org/3/library/threading.html#barrier-objects """ def __init__(self, parties: int) -> None: self.parties = parties self._waiting: int self._event = asyncio.Event() def add_one(self) -> None: self._waiting += 1 if self._waiting == self.parties: self._event.set() async def wait(self, worker: "Worker") -> None: """ Wait until all we have at least `parties` waiters. """ self.add_one() await self._event.wait() Le jeu. 25 févr. 2021 à 16:42, Barry Scott a écrit :

...

On 25 Feb 2021, at 13:14, Yves Duprat wrote:

Hi,the list,

I'm wondering why Barrier object does not exist in the synchronization primitives of the asyncio lib while it is present in threading and multiprocessing libs ? This may not be the right place to ask this question, but I never found an answer on the web. Thanks for your help.

I'm assuming that the barrier you are speaking of is the mechanism that is used to synchronise threads/processes running in parallel to prevent data races.

With async code that is never an issue. Each function runs to completion uninterrupted. There are no data races. Each time a async function runs it can know that the state of the objects it uses will not be changed while it is running.

Barry

...

Yves _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at

https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7...

...

Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPX... Code of Conduct: http://python.org/psf/codeofconduct/

On 25 Feb 2021, at 17:15, Jonathan Slenders wrote:

It does make sense to have a barrier synchronization primitive for asyncio. The idea is to make a coroutine block until at least X coroutines are waiting to enter the barrier. This is very useful, if certain actions need to be synchronized.

I do most of my async coding with twisted where what you calling a barrier is a DeferredList. The way its used is that you add in all the deferreds that you want to complete before you continue into the list. Once all the deferered have competed the DefferedList completes and its callback is run. Barry

Recently, I had to implement a barier myself for our use case. See code below:

It is simple to implement, but I too would like to have one for asyncio, in order to be consistent with the concurrency primitives we have for threading.

Jonathan

class Barier: """ Make a coroutine block until there are at least X waiters.

Similar to the threading Barier objects but for asyncio: https://docs.python.org/3/library/threading.html#barrier-objects https://docs.python.org/3/library/threading.html#barrier-objects """

def __init__(self, parties: int) -> None: self.parties = parties self._waiting: int self._event = asyncio.Event()

def add_one(self) -> None: self._waiting += 1 if self._waiting == self.parties: self._event.set()

async def wait(self, worker: "Worker") -> None: """ Wait until all we have at least `parties` waiters. """ self.add_one() await self._event.wait()

Le jeu. 25 févr. 2021 à 16:42, Barry Scott mailto:barry@barrys-emacs.org> a écrit :

...

On 25 Feb 2021, at 13:14, Yves Duprat mailto:yduprat@gmail.com> wrote:

Hi,the list,

I'm wondering why Barrier object does not exist in the synchronization primitives of the asyncio lib while it is present in threading and multiprocessing libs ? This may not be the right place to ask this question, but I never found an answer on the web. Thanks for your help.

I'm assuming that the barrier you are speaking of is the mechanism that is used to synchronise threads/processes running in parallel to prevent data races.

With async code that is never an issue. Each function runs to completion uninterrupted. There are no data races. Each time a async function runs it can know that the state of the objects it uses will not be changed while it is running.

Barry

...

Yves _______________________________________________ Python-ideas mailing list -- python-ideas@python.org mailto:python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org mailto:python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7... https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7... Code of Conduct: http://python.org/psf/codeofconduct/ http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org mailto:python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org mailto:python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPX... https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPX... Code of Conduct: http://python.org/psf/codeofconduct/ http://python.org/psf/codeofconduct/

On 26 Feb 2021, at 08:31, Jonathan Slenders wrote:

Barry,

What you describe sounds like `asyncio.gather(...)` if I understand correctly.

The thing with a Barier is that it's usable in situations where we don't know the other tasks. Maybe there is no reference to them from the current scope. Maybe they are even not yet created. It certainly can be done with a list of `asyncio.Future` and `asyncio.gather(...)`, but that's a lot of boilerplate.

IMHO, Yves is right. For both asyncio and threading, we have Lock, Event, Condition, Semaphore and BoundedSemaphore. Only Barier is missing among the asyncio primitives. (RLock doesn't make sense.)

Why would you need locks for async? Is it to sync with things outside of the async process? With the large and complex async app I work on there are no locks at all.

(I guess we can probably go to bugs.python.org http://bugs.python.org/ with this proposal.)

Having shown that a Barrier for async is a missing piece it would be good to get a thumbs up here. Barry

Jonathan

Le jeu. 25 févr. 2021 à 23:38, Barry Scott mailto:barry@barrys-emacs.org> a écrit :

...

On 25 Feb 2021, at 17:15, Jonathan Slenders mailto:jonathan@slenders.be> wrote:

It does make sense to have a barrier synchronization primitive for asyncio. The idea is to make a coroutine block until at least X coroutines are waiting to enter the barrier. This is very useful, if certain actions need to be synchronized.

I do most of my async coding with twisted where what you calling a barrier is a DeferredList.

The way its used is that you add in all the deferreds that you want to complete before you continue into the list. Once all the deferered have competed the DefferedList completes and its callback is run.

Barry

...

Recently, I had to implement a barier myself for our use case. See code below:

It is simple to implement, but I too would like to have one for asyncio, in order to be consistent with the concurrency primitives we have for threading.

Jonathan

class Barier: """ Make a coroutine block until there are at least X waiters.

Similar to the threading Barier objects but for asyncio: https://docs.python.org/3/library/threading.html#barrier-objects https://docs.python.org/3/library/threading.html#barrier-objects """

def __init__(self, parties: int) -> None: self.parties = parties self._waiting: int self._event = asyncio.Event()

def add_one(self) -> None: self._waiting += 1 if self._waiting == self.parties: self._event.set()

async def wait(self, worker: "Worker") -> None: """ Wait until all we have at least `parties` waiters. """ self.add_one() await self._event.wait()

Le jeu. 25 févr. 2021 à 16:42, Barry Scott mailto:barry@barrys-emacs.org> a écrit :

...

On 25 Feb 2021, at 13:14, Yves Duprat mailto:yduprat@gmail.com> wrote:

Hi,the list,

I'm wondering why Barrier object does not exist in the synchronization primitives of the asyncio lib while it is present in threading and multiprocessing libs ? This may not be the right place to ask this question, but I never found an answer on the web. Thanks for your help.

I'm assuming that the barrier you are speaking of is the mechanism that is used to synchronise threads/processes running in parallel to prevent data races.

With async code that is never an issue. Each function runs to completion uninterrupted. There are no data races. Each time a async function runs it can know that the state of the objects it uses will not be changed while it is running.

Barry

...

Yves _______________________________________________ Python-ideas mailing list -- python-ideas@python.org mailto:python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org mailto:python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7... https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7... Code of Conduct: http://python.org/psf/codeofconduct/ http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org mailto:python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org mailto:python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPX... https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPX... Code of Conduct: http://python.org/psf/codeofconduct/ http://python.org/psf/codeofconduct/

Could somone provide a concrete example on a proposed use of such an asyncio.Barrier? On Fri, 2021-02-26 at 14:19 -0800, Guido van Rossum wrote:

On Fri, Feb 26, 2021 at 10:09 AM Yves Duprat wrote:

...

I was expecting an explanation about the initial request. Is there an oversight (??) or an another reason to not have a Barrier primitive in asyncio ?

Probably because nobody working on asyncio at the time had any experience using threading.Barrier. I know that I have used all the other primitives, but I don't recall ever having used Barrier. I don't think there's anything deeper than that. You should probably just open a bpo issue (mention this thread) and submit a PR linked there. I can't promise to review it though. :-)   _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GNUCKA... Code of Conduct: http://python.org/psf/codeofconduct/