On Mon, Jun 26, 2017 at 10:02 AM, Dima Tisnek <dimaqq@gmail.com> wrote:
Chris, coming back to your use-case. Do you want to synchronise side-effect creation/deletion for the sanity of side-effects only? Or do you imply that callers' actions are synchronised too? In other words, do your callers use those directories out of band?
If I understand your question, the former. The callers aren't / need not be synchronized, and they aren't aware of the underlying synchronization happening inside the higher-level create() and delete() functions they would be using. (These are the two higher-level functions described in my pseudocode.) The synchronization is needed inside these create() and delete() functions since the low-level directory operations occur in different threads (because they are wrapped by run_in_executor()). --Chris
P.S./O.T. when it comes to directories, you probably want hierarchical locks rather than RW.
On 26 June 2017 at 11:28, Chris Jerdonek <chris.jerdonek@gmail.com> wrote:
On Mon, Jun 26, 2017 at 1:43 AM, Dima Tisnek <dimaqq@gmail.com> wrote:
Perhaps you can share your use-case, both as pseudo-code and a link to real code.
I'm specifically interested to see why/where you'd like to use a read-write async lock, to evaluate if this is something common or specific, and if, perhaps, some other paradigm (like queue, worker pool, ...) may be more useful in general case.
I'm also curious if a full set of async sync primitives may one day lead to async monitors. Granted, simple use of async monitor is really a future/promise, but perhaps there are complex use cases in the UI/react domain with its promise/stream dichotomy.
Thank you, Dima. In my last email I shared pseudo-code for an approach to read-write synchronization that is independent of use case. [1]
For the use case, my original purpose in mind was to synchronize many small file operations on disk like creating and removing directories that possibly share intermediate segments. The real code isn't public. But these would be operations like os.makedirs() and os.removedirs() that would be wrapped by loop.run_in_executor() to be non-blocking. The directory removal using os.removedirs() is the operation I thought should require exclusive access, so as not to interfere with directory creations in progress.
Perhaps a simpler, dirtier approach would be not to synchronize at all and simply retry directory creations that fail until they succeed. That could be enough to handle rare cases where simultaneous creation and removal causes an error. You could view this an EAFP approach.
Either way, I think the process of thinking through patterns for read-write synchronization is helpful for getting a better general feel and understanding of async.
--Chris
Cheers, d.
On 25 June 2017 at 23:13, Chris Jerdonek <chris.jerdonek@gmail.com> wrote:
I'm relatively new to async programming in Python and am thinking through possibilities for doing "read-write" synchronization.
I'm using asyncio, and the synchronization primitives that asyncio exposes are relatively simple [1]. Have options for async read-write synchronization already been discussed in any detail?
I'm interested in designs where "readers" don't need to acquire a lock -- only writers. It seems like one way to deal with the main race condition I see that comes up would be to use loop.time(). Does that ring a bell, or might there be a much simpler way?
Thanks, --Chris
[1] https://docs.python.org/3/library/asyncio-sync.html _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/