<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Oct 17, 2013 at 3:51 PM, Oscar Benjamin <span dir="ltr"><<a href="mailto:oscar.j.benjamin@gmail.com" target="_blank">oscar.j.benjamin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 17 October 2013 20:01, Guido van Rossum <<a href="mailto:guido@python.org">guido@python.org</a>> wrote:<br>
> On Thu, Oct 17, 2013 at 11:55 AM, Oscar Benjamin<br>
> <<a href="mailto:oscar.j.benjamin@gmail.com">oscar.j.benjamin@gmail.com</a>> wrote:<br>
>><br>
>> On 17 October 2013 19:40, Xavier Morel <<a href="mailto:python-dev@masklinn.net">python-dev@masklinn.net</a>> wrote:<br>
>> > I think there's already a significant split between context managers<br>
>> > which handle the lifecycle of a local resource (file, transaction) and<br>
>> > those which purport to locally alter global-ish state (cwd,<br>
>> > decimal.localcontext, logging.captureWarnings, redirect_stdout).<br>
>> ><br>
>> > And the latter worries me (much more than the very localized behavior of<br>
>> > suppress) because I don't see any way to implement them safely and<br>
>> > correctly when mixing it with coroutines in today's Python (some of them<br>
>> > aren't even thread-safe), all of that while I expect coroutines will see<br>
>> > significantly more use in the very near future with yield from and<br>
>> > tulip's promotion of coroutine-style async.<br>
>><br>
>> I maybe misunderstanding how the coroutine-style async works but I<br>
>> would have thought that it would be as simple as: don't use<br>
>> global-state-restoring-context-managers around statements that yield<br>
>> control (it would be simpler if there was a good term for describing<br>
>> that kind of CM). That's simpler to implement and computationally<br>
>> cheaper than e.g. the thread-local state used by the decimal module.<br>
><br>
> Context managers that actually save and restore *global* state are already<br>
> not thread-safe, so concluding they are also not coroutine-safe (or<br>
> task-safe?) seems a small step.<br>
><br>
> I'd be more worried about context manager that use thread-local state --<br>
> there is no similar concept in Tulip.<br>
<br>
</div></div>It's unnecessary in Tulip. The need for thread-local state in e.g.<br>
decimal contexts is driven by the fact that multi-threaded execution<br>
switches in an uncontrollable way. Tulip specifically makes it<br>
possible to control the points at which a switch occurs making this<br>
safe (even if localcontext() wasn't thread-safe):<br>
<br>
with decimal.localcontext() as ctx:<br>
ctx.prec = 100<br>
c = a + b<br>
# more synchronous decimal calculations<br>
<br>
# State is restored before allowing other code to execute<br>
yield from save_in_database(c)<br>
<br>
So it's fine to use global/thread-local state modifying/restoring<br>
context managers in Tulip as long as you don't yield control to other<br>
code within the with block. (unless I misunderstand - I lost track of<br>
Tulip some time ago).<br></blockquote><div><br></div><div>You've got it exactly right.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The issue with decimal.localcontext() and yield arises when using<br>
generators as much as coroutines e.g.:<br>
<br>
def exact_sum(nums):<br>
start = Decimal(0)<br>
with decimal.localcontext() as ctx:<br>
ctx.traps[decimal.Inexact] = True<br>
for num in nums:<br>
try:<br>
total += Decimal(num)<br>
except decimal.Inexact:<br>
ctx.prec *= 2<br>
return total<br>
<br>
The above is fine for computing the sum of a list of<br>
Decimals/ints/floats. However it fails if you pass in a generator that<br>
carelessly modifies the arithmetic context around yield calls:<br>
<br>
def compute():<br>
with decimal.localcontext() as ctx:<br>
ctx.prec = 15<br>
ctx.traps[decimal.Inexact] = False<br>
yield a + b<br>
yield b - c<br>
# etc.<br>
<br>
exact_sum(compute())<br>
<br>
There needs to be a convention that either functions like exact_sum()<br>
mustn't assume continuity of the context between iterations or a<br>
function like compute() must restore before yielding. IMO the only<br>
sane approach for async coroutines is to say that if you yield or<br>
yield from then it is your responsibility to restore any temporarily<br>
altered global/thread-local state first.<span class="HOEnZb"></span><br></blockquote></div><br></div><div class="gmail_extra">Right again. The simplest rule to remember seems to be "don't use yield or yield-from inside a with-statement". You can relax it by limiting it to context managers that manage any kind of shared resource, but that is probably already too subtle: e.g. yielding inside "with open(file) as f" seems fine, but yielding inside "with lock" is problematic, since the other side might try to acquire the same lock, and deadlock.<br clear="all">
</div><div class="gmail_extra"><br>-- <br>--Guido van Rossum (<a href="http://python.org/~guido">python.org/~guido</a>)
</div></div>