Jim Jewett wrote:
On 3/22/07, Ron Adam <rrr@ronadam.com> wrote:
Josiah Carlson wrote:
Ron Adam <rrr@ronadam.com> wrote:
Josiah Carlson wrote:
On the upside, this is possible (change the PyObject_HEAD macro, PyINCREF, PyDECREF, remove the GIL), but the amount of work
The zillions of details are the same details you have to consider when writing a concurrent database. Having python store its objects externally and retrieve them when needed (vs allocating memory and pointers) would be a huge one-time loss, but it might eventually be worthwhile.
PyPy calls that "external database" an "object_space", so the language support is already there, and still will be when the hardware makes it worthwhile.
(1) xyz = vector(r) forall coords as c: # parallel modify coords 'inplace' with body c = rotate(c, xyz)
* 'with' syntax form because 'c' does not outlive the body.
Why not just keep using
xyz = vector(r) rotate(xyz)
and let the compiler take care of it?
Just how much should the 'python' compiler do? And what about dynamic modeling where things change according to input? (Not just thinking of graphics.)
At most, we would want a way of marking objects as "read-only" or callables as "instant", so that the compiler would know it doesn't have to worry about the definition of "len" changing mid-stream. (Ron suggests something similar at the end of the message, and Talin's Transactional metaclass is related.)
Would it be possible to have a container for which it's contents are read only? (as a side effect of being in the container) Then individual items would not need their own read only attributes. A few other thoughts... Possibly ... when a module is first imported or ran nothing needs to be marked read only. Then when execution falls off the end, *everything* existing at that point is marked read only. If the module was run from as a script, then a main function is executed. These could be optional settings that could be turned on... from __far_future__ import multi-processing __multi-processing__ = True __main__ = "my_main" So then there would be an initiation first pass followed by an execution phase. In the initiation phase it's pretty much just the way things are now except you can't use threads. In the execution phase you can use threads, but you can't change anything created in the initiation phase. (Other controls would still be needed of course.)
* Generators with the semantics of calculating first and waiting on 'yield' for 'next', so the value is immediately returned. (depends on CPU load)
On its own, this just makes response time worse.
Yes, you really wouldn't use these for very simple counters. For more complex things, the calling overhead becomes a much smaller percentage of the total, and it would have a bigger effect on smoothing out response time rather than hurting it.
That said, generators are also callables, and the compiler might do a better job if it knew that it wouldn't have to worry about external redefinitions.
There may also be some value in some sort of "idletasks" abstraction that says "hey, go ahead and precompute this, but only if you've got nothing better to do."
Having a way to set process priority may be good. (or bad)
Garbage collection could certainly benefit from this, and translating (or duplicating) string representations into multiple encodings.
In the video he also talked about avoiding locks. I was thinking a more limited function object (for concurrent uses only) might be used that has:
* no global keyword * only access to external immutable objects
Or at least, it doesn't mutate them itself, and it doesn't promise to use newer versions that get created after the call begins.
The difficulty is that even a method call on a global (or parent scope) object can result in it being mutated. So you are either back to using locks and/or transactions on everything.
* no closures.
This level of information should be useful.
But in practice, most of my functions already meet this definition.
Mine too. Ron
(In theory, they access builtins that could be replaced, etc.) I wouldn't want to mark them all by hand. I'm still inclined to trust the compiler, and just accept that it may eventually be the PyPy translator rather than the CPython interpreter.
-jJ