[Async-sig] [python-tulip] asyncio.Lock equivalent for multiple processes

Nickolai Novik nickolainovik at gmail.com
Tue Apr 17 06:46:10 EDT 2018

Hi, redis lock has own limitations and depending on your use case it may or
may not be suitable [1]. If possible I would redefine problem and also
1) create worker per specific resource type to avoid locking
2) optimistic locking
3) File system lock like in twisted, but not sure about performance and
edge cases there
4) make operation on resource idempotent


On Tue, 17 Apr 2018 at 13:01 Ludovic Gasc <gmludo at gmail.com> wrote:

> Hi Roberto,
> Thanks for the pointer, it's exactly the type of feedbacks I'm looking
> for: Ideas that are out-of-box of my confort zone.
> However, in our use case, we are using gunicorn, that uses forks instead
> of multiprocessing to my knowledge, I can't use multiprocessing without to
> remove gunicorn.
> If somebody is using aioredlock in his project, I'm interested by
> feedbacks.
> Have a nice week.
> --
> Ludovic Gasc (GMLudo)
> 2018-04-17 7:19 GMT+02:00 Roberto Martínez <robertomartinezp at gmail.com>:
>> Hi,
>> I don't know if there is a third party solution for this.
>> I think the closest you can get today using the standard library is using
>> a multiprocessing.manager().Lock (which can be shared among processes) and
>> call the lock.acquire() function with asyncio.run_in_executor(), using a
>> ThreadedPoolExecutor to avoid blocking the asyncio event loop.
>> Best regards,
>> Roberto
>> El mar., 17 abr. 2018 a las 0:05, Ludovic Gasc (<gmludo at gmail.com>)
>> escribió:
>>> Hi,
>>> I'm looking for a equivalent of asyncio.Lock (
>>> https://docs.python.org/3/library/asyncio-sync.html#asyncio.Lock) but
>>> shared between several processes on the same server, because I'm migrating
>>> a daemon from mono-worker to multi-worker pattern.
>>> For now, the closest solution in term of API seems aioredlock:
>>> https://github.com/joanvila/aioredlock#aioredlock
>>> But I'm not a big fan to use polling nor with a timeout because the lock
>>> I need is very critical, I prefer to block the code than unlock with
>>> timeout.
>>> Do I miss a new awesome library or do you have an easier approach ?
>>> Thanks for your responses.
>>> --
>>> Ludovic Gasc (GMLudo)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20180417/8b3c2f09/attachment.html>

More information about the Async-sig mailing list