I have to assume the following is either not useful, not practical, or both -but I have to ask... Version I. It seems to me that PyPy might be useful as a kind of "Co-Python" -viz. the PyPy VM running as a CPython extension, as a way to utilise a second core from a single CPython process. I'm realistic about the level of integration that could be achieved, -but even something that amounted to (fast) RPC inside a process might be useful given the second vm was arguably "free for the using" on a multicore system. If this was in fact useful, it might be an on-ramp into the Python mainstream for PyPy. PyPy needs on-ramps imho. Version II. Is "multiple isolated (but bridged) VM's in a single process" a possible general technique? If multiple cooperating processes is a reasonable response to the "single threaded VM meets multi-core architecture" problem -then is "multiple VM's in a single process" a better one? I'm talking about an ugly, brute force alternative to a multi threaded VM. Viz. simply duplicating / munging / mangling / tagging as much of the interpreter code as was necessary to allow say two (but not n) threads to safely run at once. 2 heaps, 2 GILS and any given thread only ever running against one of these "sub vm's". Then perhaps add explicitly managed opportunities for memory sharing and other integration strategies betwen the two. It strikes me as something that you wouldn't want to attempt in C, but might be possible in the PyPy translation. "Dual-core Python" .."Hydra" -or is this just crazy talk? Pete F
Peter Fraser wrote:
-or is this just crazy talk?
Very confused at many levels. Putting aside the relevance to pypy, you seem to confuse VM designs and VM instances, in the sense that for some mysterious reason you expect VM instances to play together fantastically better if they are of different designs. But AFAIK, identical twins tend to sync rather better than arbitrary brothers or sisters...
* Peter Fraser <pfraser@spatialmedia.com> [2007-11-16 22:53:39 +1300]:
It seems to me that PyPy might be useful as a kind of "Co-Python" -viz. the PyPy VM running as a CPython extension, as a way to utilise a second core from a single CPython process.
I'm realistic about the level of integration that could be achieved, -but even something that amounted to (fast) RPC inside a process might be useful given the second vm was arguably "free for the using" on a multicore system.
Why not just run two CPython VMs, instead of one CPython VM and one PyPy VM?
If multiple cooperating processes is a reasonable response to the "single threaded VM meets multi-core architecture" problem -then is "multiple VM's in a single process" a better one?
If you're running multiple VMs, and you want to take advantage of multi-core architecture, running them in multiple processes seems like a much better idea than running them in separate threads in the same process. -- mithrandi, i Ainil en-Balandor, a faer Ambar
Hi Peter, On Fri, Nov 16, 2007 at 10:53:39PM +1300, Peter Fraser wrote:
Is "multiple isolated (but bridged) VM's in a single process" a possible general technique?
If multiple cooperating processes is a reasonable response to the "single threaded VM meets multi-core architecture" problem -then is "multiple VM's in a single process" a better one?
This is probably yet another place where PyPy could go much more easily than CPython: multiple VMs in the same process which would by default not have any share state, but expose primitives to the Python programmer to explicitly share some objects. This is not trivial: there is some research and experimentation to be done here. Right now, the main issue with that is that it's definitely "yet another cool thing that we could do with PyPy". As was discussed at length, in the current situation the core team's feeling is that it needs to focus on providing a useful "baseline" Python interpreter. Someone else is welcome to jump in and play, though. A bientot, Armin.
First I should probably introduce myself. I've been lurking on the pypy-dev list for about 6 months now, ever since I discovered the project. I think PyPy looks like it has truly awe-inspiring potential, and I'm just about getting to the point where I am able to play around with it and maybe think about contributing a bit. Sadly sprints are a bit hard to get to from Australia though. So yes. Name's Ben. *tips hat* Quoting Armin Rigo <arigo@tunes.org>:
Hi Peter,
On Fri, Nov 16, 2007 at 10:53:39PM +1300, Peter Fraser wrote:
Is "multiple isolated (but bridged) VM's in a single process" a possible general technique?
If multiple cooperating processes is a reasonable response to the "single threaded VM meets multi-core architecture" problem -then is "multiple VM's in a single process" a better one?
This is probably yet another place where PyPy could go much more easily than CPython: multiple VMs in the same process which would by default not have any share state, but expose primitives to the Python programmer to explicitly share some objects. This is not trivial: there is some research and experimentation to be done here.
Right now, the main issue with that is that it's definitely "yet another cool thing that we could do with PyPy". As was discussed at length, in the current situation the core team's feeling is that it needs to focus on providing a useful "baseline" Python interpreter. Someone else is welcome to jump in and play, though.
This is an area I would love to work on, but I think it's a bit beyond my capabilities just yet. I have a much more modest project I'd like to do with pypy more immediately. I was planning on posting here for advice after I've poked around a bit (haven't yet had time to really do anything since I did my first svn checkout the other day), in fact. I want to see if I can use pypy to make a python shell, both to become more familliar with pypy's internals and because I want one. :) But anyway, one of the things I would like to work on in the future if I get a bit more expertise is making pypy "multi-core friendly", so I guess if there are any ideas already floating around of lines of experimentation to try, I certainly would like to hear them. -- Ben Mellor ------------------------------------------------------------ This email was sent from Netspace Webmail: http://www.netspace.net.au
participants (5)
-
Armin Rigo -
Boris Borcic -
cumber＠netspace.net.au -
Peter Fraser -
Tristan Seligmann