Would comments from a project using this approach in real systems be of interest/use/help? Whilst I didn't know about Morrison's FBP (Balzer's work predates him btw - don't listen to hype) I had heard of (and played with) Occam among other more influential things, and Kamaelia is a real tool. Also there is already a pre-existing FBP tool for Stackless, and then historically there's also MASCOT & friends. It just looks to me that you're tieing yourself up in knots over things that aren't problems, when there are some things which could be useful (in practice) & interesting in this space. Oh, incidentally, Mini Axon is a toy/teaching/testing system - as the name suggests. The main Axon is more complete -- in the areas we've needed - it's been driven by real system needs. (for those who don't know me, Kamaelia is my project, I don't bite, but I do sometimes talk or type fast) Regards, Michael Sparks -- http://www.kamaelia.org/PragmaticConcurrency.html http://yeoldeclue.com/blog On 7/29/10, Kevin Ar18 <kevinar18@hotmail.com> wrote:
As a followup to my earlier post: "pre-emptive micro-threads utilizing shared memory message passing?"
I am actually finding that the biggest hurdle to accomplishing what I want is the lack of ANY type of shared memory -- even if it is limited. I wonder if I might ask a question:
Would the following be a possible way to offer a limited type of shared memory:
Summary: create a system very, very similar to POSH, but with differences:
In detail, here's what I mean: * unlike POSH, utilize OS threads and shared memory (not processes) * Create a special shared memory location where you can place Python objects * Each Python object you place into this location can only be accessed (modified) by 1 thread. * You must manually assign ownership of an object to a particular thread. * The thread that "owns" the object is the only one that can modify it. * You can transfer ownership to another thread (but, as always only the owner can modify it).
* There is no GIL when a thread interacts with these special objects. You can have true thread parallelism if your code uses a lot of these special objects. * The GIL remains in place for all other data access. * If your code has a mixture of access to the special objects and regular data, then once you hit a point where a thread starts to interact with data not in the special storage, then that thread must follow GIL rules.
Granted, there might be some difficulty with the GIL part... but I thought I might ask anyways. :)
Date: Wed, 28 Jul 2010 22:54:39 +1000 Subject: Re: [pypy-dev] pre-emptive micro-threads utilizing shared memory message passing? From: william.leslie.ttg@gmail.com To: kevinar18@hotmail.com CC: pypy-dev@codespeak.net
On 28 July 2010 04:20, Kevin Ar18 <kevinar18@hotmail.com> wrote:
I am attempting to experiment with FBP - Flow Based Programming (http://www.jpaulmorrison.com/fbp/ and book: http://www.jpaulmorrison.com/fbp/book.pdf) There is something very similar in Python: http://www.kamaelia.org/MiniAxon.html Also, there are some similarities to Erlang - the share nothing memory model... and on some very broad levels, there are similarities that can be found in functional languages.
Does anyone know if there is a central resource for incompatible python memory model proposals? I know of Jython, Python-Safethread, and Mont-E.
I do like the idea of MiniAxon, but let me mention a topic that has slowly been bubbling to the front of my mind for the last few months.
Concurrency in the face of shared mutable state is hard. It makes it trivial to introduce bugs all over the place. Nondeterminacy related bugs are far harder to test, diagnose, and fix than anything else that I would almost mandate static verification (via optional typing, probably) of task noninterference if I was moving to a concurrent environment with shared mutable state. There might be a reasonable middle ground where, if a task attempts to violate the required static semantics, it fails dynamically. At least then, latent bugs make plenty of noise. An example for MiniAxon (as I understand it, which is not very well) would be verification that a "task" (including functions that the task calls) never closes over and yields the same mutable objects, and never mutates globally reachable objects.
I wonder if you could close such tasks off with a clever subclass of the proxy object space that detects and rejects such memory model violations? With only semantics that make the program deterministic?
The moral equivalent would be cooperating processes with a large global (or not) shared memory store for immutable objects, queues for communication, and the additional semantic that objects in a queue are either immutable or the queue holds their only reference. The trouble is that it is so hard to work out what immutable really means. Non-optional annotations would be not very pythonian.
-- William Leslie