[Async-sig] "read-write" synchronization

[Dima Tisnek]

A little epiphany on my part:

In threaded world, a lock (etc.) can be used for 2 distinct purposes:
*1 synchronise [access to resource in the] library implementation, and
*2 synchronise users of a library

It's easy since taken lock has an owner (thread).
Both library and user stack frames belong to either this thread or some other.


In the async world, users are opaque to library implementation
(technically own async threads).
Therefore only use case #1 is valid.
Moreover, it occurs to me that lock/unlock pair must be confined to
same async function.
Going beyond that restriction is bug-prone like crazy (even for me).


Chris, coming back to your use-case.
Do you want to synchronise side-effect creation/deletion for the
sanity of side-effects only?
Or do you imply that callers' actions are synchronised too?
In other words, do your callers use those directories out of band?


P.S./O.T. when it comes to directories, you probably want hierarchical
locks rather than RW.


On 26 June 2017 at 11:28, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> On Mon, Jun 26, 2017 at 1:43 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
>> Perhaps you can share your use-case, both as pseudo-code and a link to
>> real code.
>>
>> I'm specifically interested to see why/where you'd like to use a
>> read-write async lock, to evaluate if this is something common or
>> specific, and if, perhaps, some other paradigm (like queue, worker
>> pool, ...) may be more useful in general case.
>>
>> I'm also curious if a full set of async sync primitives may one day
>> lead to async monitors. Granted, simple use of async monitor is really
>> a future/promise, but perhaps there are complex use cases in the
>> UI/react domain with its promise/stream dichotomy.
>
> Thank you, Dima. In my last email I shared pseudo-code for an approach
> to read-write synchronization that is independent of use case. [1]
>
> For the use case, my original purpose in mind was to synchronize many
> small file operations on disk like creating and removing directories
> that possibly share intermediate segments. The real code isn't public.
> But these would be operations like os.makedirs() and os.removedirs()
> that would be wrapped by loop.run_in_executor() to be non-blocking.
> The directory removal using os.removedirs() is the operation I thought
> should require exclusive access, so as not to interfere with directory
> creations in progress.
>
> Perhaps a simpler, dirtier approach would be not to synchronize at all
> and simply retry directory creations that fail until they succeed.
> That could be enough to handle rare cases where simultaneous creation
> and removal causes an error. You could view this an EAFP approach.
>
> Either way, I think the process of thinking through patterns for
> read-write synchronization is helpful for getting a better general
> feel and understanding of async.
>
> --Chris
>
>
>>
>> Cheers,
>> d.
>>
>> On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
>>> I'm relatively new to async programming in Python and am thinking
>>> through possibilities for doing "read-write" synchronization.
>>>
>>> I'm using asyncio, and the synchronization primitives that asyncio
>>> exposes are relatively simple [1]. Have options for async read-write
>>> synchronization already been discussed in any detail?
>>>
>>> I'm interested in designs where "readers" don't need to acquire a lock
>>> -- only writers. It seems like one way to deal with the main race
>>> condition I see that comes up would be to use loop.time(). Does that
>>> ring a bell, or might there be a much simpler way?
>>>
>>> Thanks,
>>> --Chris
>>>
>>>
>>> [1] https://docs.python.org/3/library/asyncio-sync.html
>>> _______________________________________________
>>> Async-sig mailing list
>>> Async-sig at python.org
>>> https://mail.python.org/mailman/listinfo/async-sig
>>> Code of Conduct: https://www.python.org/psf/codeofconduct/