[pypy-dev] Would the following shared memory model be possible?

[Michael Sparks]

Would comments from a project using this approach in real systems be
of interest/use/help? Whilst I didn't know about Morrison's FBP
(Balzer's work predates him btw - don't listen to hype) I had heard of
(and played with) Occam among other more influential things, and
Kamaelia is a real tool. Also there is already a pre-existing FBP tool
for Stackless, and then historically there's also MASCOT & friends. It
just looks to me that you're tieing yourself up in knots over things
that aren't problems, when there are some things which could be useful
(in practice) & interesting in this space.

Oh, incidentally, Mini Axon is a toy/teaching/testing system - as the
name suggests. The main Axon is more complete -- in the areas we've
needed - it's been driven by real system needs.

(for those who don't know me, Kamaelia is my project, I don't bite,
but I do sometimes talk or type fast)

Regards,

Michael Sparks
--
http://www.kamaelia.org/PragmaticConcurrency.html
http://yeoldeclue.com/blog

On 7/29/10, Kevin Ar18 <kevinar18 at hotmail.com> wrote:
> As a followup to my earlier post:
> "pre-emptive micro-threads utilizing shared memory message passing?"
>
> I am actually finding that the biggest hurdle to accomplishing what I want
> is the lack of ANY type of shared memory -- even if it is limited.  I wonder
> if I might ask a question:
>
> Would the following be a possible way to offer a limited type of shared
> memory:
>
> Summary: create a system very, very similar to POSH, but with differences:
>
> In detail, here's what I mean:
> * unlike POSH, utilize OS threads and shared memory (not processes)
> * Create a special shared memory location where you can place Python objects
> * Each Python object you place into this location can only be accessed
> (modified) by 1 thread.
> * You must manually assign ownership of an object to a particular thread.
> * The thread that "owns" the object is the only one that can modify it.
> * You can transfer ownership to another thread (but, as always only the
> owner can modify it).
>
> * There is no GIL when a thread interacts with these special objects.  You
> can have true thread parallelism if your code uses a lot of these special
> objects.
> * The GIL remains in place for all other data access.
> * If your code has a mixture of access to the special objects and regular
> data, then once you hit a point where a thread starts to interact with data
> not in the special storage, then that thread must follow GIL rules.
>
> Granted, there might be some difficulty with the GIL part... but I thought I
> might ask anyways. :)
>
>> Date: Wed, 28 Jul 2010 22:54:39 +1000
>> Subject: Re: [pypy-dev] pre-emptive micro-threads utilizing shared memory
>> 	message passing?
>> From: william.leslie.ttg at gmail.com
>> To: kevinar18 at hotmail.com
>> CC: pypy-dev at codespeak.net
>>
>> On 28 July 2010 04:20, Kevin Ar18 <kevinar18 at hotmail.com> wrote:
>> > I am attempting to experiment with FBP - Flow Based Programming
>> > (http://www.jpaulmorrison.com/fbp/ and book:
>> > http://www.jpaulmorrison.com/fbp/book.pdf)  There is something very
>> > similar in Python: http://www.kamaelia.org/MiniAxon.html  Also, there
>> > are some similarities to Erlang - the share nothing memory model... and
>> > on some very broad levels, there are similarities that can be found in
>> > functional languages.
>>
>> Does anyone know if there is a central resource for incompatible
>> python memory model proposals? I know of Jython, Python-Safethread,
>> and Mont-E.
>>
>> I do like the idea of MiniAxon, but let me mention a topic that has
>> slowly been bubbling to the front of my mind for the last few months.
>>
>> Concurrency in the face of shared mutable state is hard. It makes it
>> trivial to introduce bugs all over the place. Nondeterminacy related
>> bugs are far harder to test, diagnose, and fix than anything else that
>> I would almost mandate static verification (via optional typing,
>> probably) of task noninterference if I was moving to a concurrent
>> environment with shared mutable state. There might be a reasonable
>> middle ground where, if a task attempts to violate the required static
>> semantics, it fails dynamically. At least then, latent bugs make
>> plenty of noise. An example for MiniAxon (as I understand it, which is
>> not very well) would be verification that a "task" (including
>> functions that the task calls) never closes over and yields the same
>> mutable objects, and never mutates globally reachable objects.
>>
>> I wonder if you could close such tasks off with a clever subclass of
>> the proxy object space that detects and rejects such memory model
>> violations? With only semantics that make the program deterministic?
>>
>> The moral equivalent would be cooperating processes with a large
>> global (or not) shared memory store for immutable objects, queues for
>> communication, and the additional semantic that objects in a queue are
>> either immutable or the queue holds their only reference. The trouble
>> is that it is so hard to work out what immutable really means.
>> Non-optional annotations would be not very pythonian.
>>
>> --
>> William Leslie
>