[Python-ideas] Monitor implementation for the stdlib?

[Christopher D. Leary]

I didn't mean to imply that locks /couldn't/ model any of the mentioned
functionality, just that they weren't the right tool for the job that I
was describing (which I believe is frequently occurring in multithreaded
applications -- is that in dispute?).

I debated putting the unsychronized decorator into the implementation.
On one hand, a monitor is a monitor -- it's invariant is mutual
exclusion between methods; however, on the other hand you can have
related functionality to a "bridge" monitor that doesn't require mutual
exclusion, though I figured it should probably be used very sparingly,
or the point of a monitor approaches moot -- it just appealed to me more
than forcing the equivalent functionality to move to global functions.
Perhaps the alternative is better if it makes the monitor concept
significantly weaker.

That aside, I think the synchronized decorator silly (though I may be
wrong, it happens all the time ;). I can envision very few situations
where you would only want to make a couple of your methods mutually
exclusive and none of the others. In my mind this would imply you're
either using poor encapsulation or leaving yourself prone to errors...
from a maintainability and understandability perspective I believe that
you're losing value by selectively choosing which methods should be
synchronized, whereas inheriting Monitor is a clear statement that the
instance variables are protected in a thread safe manner.

I wholly agree that locks are better suited for some (very complex xor
very simple) multithreaded situations. I would think that in the common
case where grouping data for thread safety is involved, you'll have very
few, disjoint groups, in which case monitors are what you're looking for.

I'd be interested in hearing more about your reasoning for synchronized.
The monitor "everything is mutually exclusive" seems even /more/ simple
to me than "selected functions are mutually exclusive" -- it's certainly
less to worry about going forward.

Chris

Guido van Rossum wrote:
> 2007/10/21, Christopher D. Leary <christopher.leary at cornell.edu>:
>> Though I do think that the with statement is an excellent addition to
>> the language (it avoids much of the ugliness and chance for error in
>> symmetric P and V), I don't believe that it adequately covers the use
>> case that monitors are intended for; namely, encapsulation and
>> abstraction of the mutual exclusion involved in locking around /several
>> data entities/ with /several defined functionalities/.
> 
> Locks can do that too of course.
> 
> It looks like a monitor is similar to 'synchronized' in Java except it
> appears the default (once inheriting from Monitor) is synchronized,
> and you need to use a decorator to declare an unsynchronized method.
> Have I got that right?
> 
>> Due to how frequently this pattern is encountered, I think that monitors
>> not only reduce clutter with syntactic sugar (simplicity?), but help
>> encourage a more straightforward way of thinking -- several non
>> re-entrant operations on shared data implies monitor usage, not lock
>> acquisition all over the place.
> 
> I detect a slight prejudice against lock acquisition in your wording. :-)
> 
>> Thoughts?
> 
> It seems to be a matter of granularity. With monitors, one is required
> to create a new class for each group of variables that require
> synchronization. With locks, that is not necessary. Which is better
> depends on the complexity of the group of variables and (especially)
> the operations.
> 
> A more "Pythonic" approach would be to implement an @synchronized
> decorator; Python users are more likely to be familiar with Java terms
> than with classic Hoare or P and V.
> 
> (I'm sure someone has already implemented @synchronized, but it hasn't
> made it into the standard library -- perhaps an indication that its
> importance is more theoretical than practical, perhaps simply due to a
> time lag.)
> 
> --Guido
> 
>> Chris
>>
>> Guido van Rossum wrote:
>>> Isn't this just syntactic sugar on top of recursive locks and
>>> condition variables? What's the big deal apart from conforming to a
>>> historically important API?
>>>
>>> Note that as of python 2.5, the with-statement lets you create
>>> critical sections simply by writing
>>>
>>>   with <some_lock>:
>>>     <critical_section>
>>>
>>> i.e. the acquire() and release() calls are implicit in the with-statement.
>>>
>>> --Guido
>>>
>>> 2007/10/21, Christopher D. Leary <christopher.leary at cornell.edu>:
>>>> I was surprised to find that there is no "monitor construct"
>>>> implementation for the stdlib. I wrote one that I feel is pretty
>>>> Pythonic, and I'd like it to be torn apart :)
>>>>
>>>> It's fairly well documented -- for most standard monitor use cases you
>>>> can simply inherit the Monitor class and use the Monitor-specific
>>>> condition variables. Hopefully the accompanying examples will also help
>>>> to clarify the usage. They're somewhat silly, but get the idea across.
>>>>
>>>> As an aside, because this seems to come up in every conversation I've
>>>> had about monitors in python: if I'm not mistaken, monitors are useful
>>>> with or without the GIL :)
>>>>
>>>> Monitors are nifty tools that make complex synchronization problems
>>>> somewhat simpler (though more serialized). So far as I understand it,
>>>> the GIL provides single-bytecode atomicity, and monitor methods are
>>>> rarely single instructions. Plus, Jython and IronPython don't have a
>>>> GIL, so I would argue that monitors can still be valuable to "Python the
>>>> language" even if you won't allow that they can be valuable to "Python
>>>> the standard implementation".
>>>>
>>>> Looking forward to hearing everybody's opinions.
>>>>
>>>> Cheers,
>>>>
>>>> Chris
>>>>
>>>> _______________________________________________
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> http://mail.python.org/mailman/listinfo/python-ideas
>>>>
>>>>
>>>>
>>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> http://mail.python.org/mailman/listinfo/python-ideas
>>
> 
> 

-- 
Christopher D. Leary
ECE/CS Undergraduate
Cornell University
C: 914.844.1622
christopher.leary at cornell.edu