<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Mon, Aug 21, 2017 at 12:50 PM, Yury Selivanov <span dir="ltr"><<a href="mailto:yselivanov.ml@gmail.com" target="_blank">yselivanov.ml@gmail.com</a>></span> wrote:<span class="gmail-"></span><br><span class="gmail-"></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-">
</span>Few important things (using the current PEP 550 terminology):<br>
<br>
* ExecutionContext is a *dynamic* stack of LogicalContexts.<br>
* LCs do not reference other LCs.<br>
* ContextKey.set() can only modify the *top* LC in the stack.<br>
<br>
If LC is a mutable mapping:<br>
<br>
# EC = [LC1, LC2, LC3, LC4({a: b, foo: bar})]<br>
<br>
a.set(c)<br>
# LC4 = EC.top()<br>
# LC4[a] = c<br>
<br>
# EC = [LC1, LC2, LC3, LC4({a: c, foo: bar})]<br>
<br>
If LC are implemented with immutable mappings:<br>
<br>
# EC = [LC1, LC2, LC3, LC4({a: b, foo: bar})]<br>
<br>
a.set(c)<br>
# LC4 = EC.pop()<br>
# LC4_1 = LC4.copy()<br>
# LC4_1[a] = c<br>
# EC.push(LC4_1)<br>
<br>
# EC = [LC1, LC2, LC3, LC4_1({a: c, foo: bar})]<br>
<br>
Any code that uses EC will not see any difference, because it can only<br>
work with the top LC.<br></blockquote><div><br></div><div>OK, good. This makes more sense, especially if I read "the EC" as shorthand for the EC stored in the current thread's per-thread state. The immutable mapping (if used) is used for the LC, not for the EC, and in this case cloning an EC would simply make a shallow copy of its underlying list -- whereas without the immutable mapping, cloning the EC would also require making shallow copies of each LC. And I guess the linked-list implementation (Approach #3 in the PEP) makes EC cloning an O(1) operation.<br></div><div><br></div><div> Note that there is a lot of hand-waving and shorthand in this explanation, but I think I finally follow the design. It is going to be a big task to write this up in a didactic way -- the current PEP needs a fair amount of help in that sense. (If you want to become a better writer, I've recently enjoyed reading Steven Pinker's <i>The Sense of Style</i>: <span class="gmail-st">The Thinking Person's Guide to Writing in the 21st Century</span>. Amongst other fascinating topics, it explains why so often what we think is clearly written can cause so much confusion.)</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Back to generators. Generators have their own empty LCs when created<br>
to store their *local* EC modifications.<br>
<br>
When a generator is *being* iterated, it pushes its LC to the EC. When<br>
the iteration step is finished, it pops its LC from the EC. If you<br>
have nested generators, they will dynamically build a stack of their<br>
LCs while they are iterated.<br>
<br>
Therefore, generators *naturally* control the stack of EC. We can't<br>
execute two generators simultaneously in one thread (we can only<br>
iterate them one by one), so the top LC always belongs to the current<br>
generator that is being iterated:<br>
<br>
def nested_gen():<br>
# EC = [outer_LC, gen1_LC, nested_gen_LC]<br>
yield<br>
# EC = [outer_LC, gen1_LC, nested_gen_LC]<br>
yield<br>
<br>
def gen1():<br>
# EC = [outer_LC, gen1_LC]<br>
n = nested_gen()<br>
yield<br>
# EC = [outer_LC, gen1_LC]<br>
next(n)<br>
# EC = [outer_LC, gen1_LC]<br>
yield<br>
next(n)<br>
# EC = [outer_LC, gen1_LC]<br>
<br>
def gen2():<br>
# EC = [outer_LC, gen2_LC]<br>
yield<br>
# EC = [outer_LC, gen2_LC]<br>
yield<br>
<br>
g1 = gen1()<br>
g2 = gen2()<br>
<br>
next(g1)<br>
next(g2)<br>
next(g1)<br>
next(g2)<br></blockquote><div><br></div><div>This, combined with your later clarification:</div><div><br></div><div>> In the current version of the PEP, generators are initialized with an<br>> empty LogicalContext. When they are being iterated (started or<br>> resumed), their LogicalContext is pushed to the EC. When the<br>> iteration is stopped (or paused), they pop their LC from the EC.</div><div><br></div><div>makes it clear how the proposal works for generators. There's an important piece that I hadn't figured out from Nick's generator example, because I had mistakenly assumed that something *would* be captured at generator create time. It's a reasonable mistake to make, I think -- the design space here is just huge and there are many variations that don't affect typical code but do differ in edge cases. Your clear statement "nothing needs to be captured" is helpful to avoid this misunderstanding.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
HAMT is a way to efficiently implement immutable mappings with O(log32<br>
N) set operation, that's it. If we implement immutable mappings using<br>
regular dicts and copy, set() would be O(log N).<br></blockquote><div><br></div><div>This sounds like abuse of the O() notation. Mathematically O(log N) and O(log32 N) surely must be equivalent, since log32 N is just K*(log N) for some constant K (about 0.288539), and the constant disappears in the O(), as O(K*f(N)) and O(f(N)) are equivalent. Now, I'm happy to hear that a HAMT-based implementation is faster than a dict+copy-based implementation, but I don't think your use of O() makes sense here.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
[..]<br>
<span class="gmail-">>><br>
>> I'll also note that the first iteration of the PEP didn't really make<br>
>> this distinction, and it caused a problem that Nathaniel pointed out:<br>
>> generators would "snapshot" their entire dynamic context when first<br>
>> created, and then never adjust it for external changes between<br>
>> iterations. This meant that if you adjusted something like the decimal<br>
>> context outside the generator after creating it, it would ignore those<br>
>> changes - instead of having the problem of changes inside the<br>
>> generator leaking out, we instead had the problem of changes outside<br>
>> the generator *not* making their way in, even if you wanted them to.<br>
><br>
><br>
> OK, this really needs to be made very clear early in the PEP. Maybe this<br>
> final sentence provides the key requirement: changes outside the generator<br>
> should make it into the generator when next() is invoked, unless the<br>
> generator itself has made an override; but changes inside the generator<br>
> should not leak out through next().<br>
<br>
</span>It's covered here with two examples:<br>
<a href="https://www.python.org/dev/peps/pep-0550/#ec-semantics-for-generators" rel="noreferrer" target="_blank">https://www.python.org/dev/<wbr>peps/pep-0550/#ec-semantics-<wbr>for-generators</a><span class="gmail-HOEnZb"></span><br></blockquote></div></div><div class="gmail_extra"><br></div><div class="gmail_extra">I think what's missing is the fact that this is one of the key motivating reasons for the design (starting with v2 of the PEP). When I encountered that section I just skimmed it, assuming it was mostly just showing how to apply the given semantics to generators. I also note some issues with the use of tense here -- it's a bit confusing to follow which parts of the text refer to defects of the current (pre-PEP) situation and which parts refer to how the proposal would solve these defects.</div><div class="gmail_extra"><br>-- <br><div class="gmail_signature">--Guido van Rossum (<a href="http://python.org/%7Eguido" target="_blank">python.org/~guido</a>)</div>
</div></div>