On 3/22/07, Ron Adam <rrr@ronadam.com> wrote:
Josiah Carlson wrote:
Ron Adam <rrr@ronadam.com> wrote:
Josiah Carlson wrote:
On the upside, this is possible (change the PyObject_HEAD macro, PyINCREF, PyDECREF, remove the GIL), but the amount of work
The zillions of details are the same details you have to consider when writing a concurrent database. Having python store its objects externally and retrieve them when needed (vs allocating memory and pointers) would be a huge one-time loss, but it might eventually be worthwhile. PyPy calls that "external database" an "object_space", so the language support is already there, and still will be when the hardware makes it worthwhile.
(1) xyz = vector(r) forall coords as c: # parallel modify coords 'inplace' with body c = rotate(c, xyz)
* 'with' syntax form because 'c' does not outlive the body.
Why not just keep using xyz = vector(r) rotate(xyz) and let the compiler take care of it? At most, we would want a way of marking objects as "read-only" or callables as "instant", so that the compiler would know it doesn't have to worry about the definition of "len" changing mid-stream. (Ron suggests something similar at the end of the message, and Talin's Transactional metaclass is related.)
* Generators with the semantics of calculating first and waiting on 'yield' for 'next', so the value is immediately returned. (depends on CPU load)
On its own, this just makes response time worse. That said, generators are also callables, and the compiler might do a better job if it knew that it wouldn't have to worry about external redefinitions. There may also be some value in some sort of "idletasks" abstraction that says "hey, go ahead and precompute this, but only if you've got nothing better to do." Garbage collection could certainly benefit from this, and translating (or duplicating) string representations into multiple encodings.
In the video he also talked about avoiding locks. I was thinking a more limited function object (for concurrent uses only) might be used that has:
* no global keyword * only access to external immutable objects
Or at least, it doesn't mutate them itself, and it doesn't promise to use newer versions that get created after the call begins.
* no closures.
This level of information should be useful. But in practice, most of my functions already meet this definition. (In theory, they access builtins that could be replaced, etc.) I wouldn't want to mark them all by hand. I'm still inclined to trust the compiler, and just accept that it may eventually be the PyPy translator rather than the CPython interpreter. -jJ