<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On 5 October 2017 at 18:45, Eric Snow <span dir="ltr"><<a href="mailto:ericsnowcurrently@gmail.com" target="_blank">ericsnowcurrently@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
After we move to not sharing the GIL between interpreters:<br>
<br>
Channel.send(obj): # in interp A<br>
incref(obj)<br>
if type(obj).tp_share == NULL:<br>
raise ValueError("not a shareable type")<br>
set_owner(obj) # obj.owner or add an obj -> interp entry to global table<br>
ch.objects.append(obj)<br>
<br>
Channel.recv(): # in interp B<br>
orig = ch.objects.pop(0)<br>
obj = orig.tp_share()<br>
set_shared(obj, orig) # add to a global table<br>
return obj<br></blockquote><div><br></div><div>This would be hard to get to work reliably, because "orig.tp_share()" would be running in the receiving interpreter, but all the attributes of "orig" would have been allocated by the sending interpreter. It gets more reliable if it's *Channel.send* that calls tp_share() though, but moving the call to the sending side makes it clear that a tp_share protocol would still need to rely on a more primitive set of "shareable objects" that were the permitted return values from the tp_share call.</div><div><br></div><div>And that's the real pay-off that comes from defining this in terms of the memoryview protocol: Py_buffer structs *aren't* Python objects, so it's only a regular C struct that gets passed across the interpreter boundary (the reference to the original objects gets carried along passively as part of the CIV - it never gets *used* in the receiving interpreter).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
bytes.tp_share():<br>
obj = blank_bytes(len(self))<br>
obj.ob_sval = self.ob_sval # hand-wavy memory sharing<br>
return obj<br></blockquote><div><br></div><div>This is effectively reinventing memoryview, while trying to pretend it's an ordinary bytes object. Don't reinvent memoryview :)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
bytes.tp_free(): # under no-shared-GIL:<br>
# most of this could be pulled into a macro for re-use<br>
orig = lookup_shared(self)<br>
if orig != NULL:<br>
current = release_LIL()<br>
interp = lookup_owner(orig)<br>
acquire_LIL(interp)<br>
decref(orig)<br>
release_LIL(interp)<br>
acquire_LIL(current)<br>
# clear shared/owner tables<br>
# clear/release self.ob_sval<br>
free(self)<br></blockquote><div><br></div>I don't think we should be touching the behaviour of core builtins solely to enable message passing to subinterpreters without a shared GIL.</div><div class="gmail_quote"><br></div><div class="gmail_quote">The simplest possible variant of CIVs that I can think of would be able to avoid that outcome by being a memoryview subclass, since they just need to hold the extra reference to the original interpreter, and include some logic to swtich interpreters at the appropriate time.</div><div class="gmail_quote"><br></div><div class="gmail_quote">That said, I think there's definitely a useful design question to ask in this area, not about bytes (which can be readily represented by a memoryview variant in the receiving interpreter), but about *strings*: they have a more complex internal layout than bytes objects, but as long as the receiving interpreter can make sure that the original string continues to exist, then you could usefully implement a "strview" type to avoid having to go through an encode/decode cycle just to pass a string to another subinterpreter.</div><div class="gmail_quote"><br></div><div class="gmail_quote">That would provide a reasonable compelling argument that CIVs *shouldn't* be implemented as memoryview subclasses, but instead defined as *containing* a managed view of an object owned by a different interpreter. <br></div><div class="gmail_quote"><br></div><div class="gmail_quote">That way, even if the initial implementation only supported CIVs that contained a memoryview instance, we'd have the freedom to define other kinds of views later (such as strview), while being able to reuse the same CIV machinery.</div><div class="gmail_quote"><br></div><div class="gmail_quote">Cheers,</div><div class="gmail_quote">Nick.<br></div><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Nick Coghlan | <a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a> | Brisbane, Australia</div>
</div></div>