<p dir="ltr">Exciting!</p>
<p dir="ltr">* <a href="http://zero-buffer.readthedocs.org/en/latest/api-reference/#zero_buffer.BufferView">http://zero-buffer.readthedocs.org/en/latest/api-reference/#zero_buffer.BufferView</a><br>
* <a href="https://www.google.com/search?q=python+channels">https://www.google.com/search?q=python+channels</a><br>
* <a href="https://docs.python.org/2/library/asyncore.html#module-asyncore">https://docs.python.org/2/library/asyncore.html#module-asyncore</a><br>
* <a href="https://chan.readthedocs.org/en/latest/">https://chan.readthedocs.org/en/latest/</a><br>
* <a href="https://goless.readthedocs.org/en/latest/">https://goless.readthedocs.org/en/latest/</a><br>
* other approaches to the problem (with great APIs):<br>
* <a href="http://celery.readthedocs.org/en/latest/userguide/canvas.html#chords">http://celery.readthedocs.org/en/latest/userguide/canvas.html#chords</a><br>
* <a href="http://discodb.readthedocs.org/en/latest/">http://discodb.readthedocs.org/en/latest/</a></p>
<div class="gmail_quote">On Jun 20, 2015 5:55 PM, "Eric Snow" <<a href="mailto:ericsnowcurrently@gmail.com">ericsnowcurrently@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr"><br>
On Jun 20, 2015 4:08 PM, "Nathaniel Smith" <<a href="mailto:njs@pobox.com" target="_blank">njs@pobox.com</a>> wrote:<br>
><br>
> On Jun 20, 2015 2:42 PM, "Eric Snow" <<a href="mailto:ericsnowcurrently@gmail.com" target="_blank">ericsnowcurrently@gmail.com</a>> wrote:<br>
> ><br>
> > tl;dr Let's exploit multiple cores by fixing up subinterpreters,<br>
> > exposing them in Python, and adding a mechanism to safely share<br>
> > objects between them.<br>
><br>
> This all sounds really cool if you can pull it off, and shared-nothing threads do seem like the least impossible model to pull off.</p>
<p dir="ltr">Agreed.</p>
<p dir="ltr">> But "least impossible" and "possible" are different :-). From your email I can't tell whether this plan is viable while preserving backcompat and memory safety.</p>
<p dir="ltr">I agree that those issues must be clearly solved in the proposal before it can be approved. I'm confident the approach I'm pursuing will afford us the necessary guarantees. I'll address those specific points directly when I can sit down and organize my thoughts.</p>
<p dir="ltr">><br>
> Suppose I have a queue between two subinterpreters, and on this queue I place a list of dicts of user-defined-in-python objects, each of which holds a reference to a user-defined-via-the-C-api object. What happens next?</p>
<p dir="ltr">You've hit upon exactly the trickiness involved and why I'm thinking the best approach initially is to only allow *strictly* immutable objects to pass between interpreters. Admittedly, my description of channels is very vague.:) There are a number of possibilities with them that I'm still exploring (CSP has particular opinions...), but immutability is a characteristic that may provide the simplest *initial* approach. Going that route shouldn't preclude adding some sort of support for mutable objects later.</p>
<p dir="ltr">Keep in mind that by "immutability" I'm talking about *really* immutable, perhaps going so far as treating the full memory space associated with an object as frozen. For instance, we'd have to ensure that "immutable" Python objects like strings, ints, and tuples do not change (i.e. via the C API). The contents of involved tuples/containers would have to be likewise immutable. Even changing refcounts could be too much, hence the idea of moving refcounts out to a separate table.</p>
<p dir="ltr">This level of immutability would be something new to Python. We'll see if it's necessary. If it isn't too much work it might be a good idea regardless of the multi-core proposal.</p>
<p dir="ltr">Also note that Barry has a (rejected) PEP from a number of years ago about freezing objects... That idea is likely out of scope as relates to my proposal, but it certainly factors in the problem space.</p>
<p dir="ltr">-eric</p>
<br>_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofconduct/</a><br></blockquote></div>