[Async-sig] "read-write" synchronization

[Chris Jerdonek]

On Mon, Jun 26, 2017 at 1:43 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
> Perhaps you can share your use-case, both as pseudo-code and a link to
> real code.
>
> I'm specifically interested to see why/where you'd like to use a
> read-write async lock, to evaluate if this is something common or
> specific, and if, perhaps, some other paradigm (like queue, worker
> pool, ...) may be more useful in general case.
>
> I'm also curious if a full set of async sync primitives may one day
> lead to async monitors. Granted, simple use of async monitor is really
> a future/promise, but perhaps there are complex use cases in the
> UI/react domain with its promise/stream dichotomy.

Thank you, Dima. In my last email I shared pseudo-code for an approach
to read-write synchronization that is independent of use case. [1]

For the use case, my original purpose in mind was to synchronize many
small file operations on disk like creating and removing directories
that possibly share intermediate segments. The real code isn't public.
But these would be operations like os.makedirs() and os.removedirs()
that would be wrapped by loop.run_in_executor() to be non-blocking.
The directory removal using os.removedirs() is the operation I thought
should require exclusive access, so as not to interfere with directory
creations in progress.

Perhaps a simpler, dirtier approach would be not to synchronize at all
and simply retry directory creations that fail until they succeed.
That could be enough to handle rare cases where simultaneous creation
and removal causes an error. You could view this an EAFP approach.

Either way, I think the process of thinking through patterns for
read-write synchronization is helpful for getting a better general
feel and understanding of async.

--Chris


>
> Cheers,
> d.
>
> On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>> I'm relatively new to async programming in Python and am thinking
>> through possibilities for doing "read-write" synchronization.
>>
>> I'm using asyncio, and the synchronization primitives that asyncio
>> exposes are relatively simple [1]. Have options for async read-write
>> synchronization already been discussed in any detail?
>>
>> I'm interested in designs where "readers" don't need to acquire a lock
>> -- only writers. It seems like one way to deal with the main race
>> condition I see that comes up would be to use loop.time(). Does that
>> ring a bell, or might there be a much simpler way?
>>
>> Thanks,
>> --Chris
>>
>>
>> [1] https://docs.python.org/3/library/asyncio-sync.html
>> _______________________________________________
>> Async-sig mailing list
>> Async-sig at python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/