Advanced concurrancy

[Peter Tillotson]

I've not yet had a chance to try some examples, but i've looked through 
the documentation. It feels quite familiar, but i'd say that it is 
closer to Jade, the fipa (federation of intelligent physical agents) 
compliant agent framework than CSP or pi calculus. I like the behaviour 
(component microthread) model, but the advantage of CSP / pi calculus is 
that the resulting distributed system remains open to mathematical analysis.

For concurency its is the best framework i've seen :-) Have you come 
across the pylinda tuplespace implementation. It might be well worth a 
look. I might be barking up the wrong tree - but it seems to me that 
there could be considerable overlap between tuplespaces and mailboxes, 
though you did mention that you were moving towards twisted as the 
underlying platform for the future.

I'm quite interested in the mini version and also using the modules as 
mobile code rather than installing it formally. I'll probably zip the 
Axion directory and distribute the zip with the code, adding the zip to 
the python path dynamically.

cheers

p

Michael Sparks wrote:
> Peter Tillotson wrote:
> 
> 
>>Hi,
>>
>>I'm looking for an advanced concurrency module for python and don't seem
>>to be able to find anything suitable. Does anyone know where I might
>>find one? I know that there is CSP like functionality built into
>>Stackless but i'd like students to be able to use a standard python build.
> 
> 
> Please take a look at Kamaelia* - it /probably/ has what you're after by the
> sounds of things. Currently the unit for sequential process can be either
> generators or threads, and is single CPU, single process, however we do
> expect to make the system multi-process and multi-system.
>    * http://kamaelia.sourceforge.net/
> 
> Currently it runs on Linux, Mac OS X, Windows and a subset works nicely on
> Series 60 mobiles. (That has separate packaging) It works with standard
> Python versions 2.2 and upwards.
> 
> The basic idea in Kamaelia is you have a class that represents a concurrent
> unit that communicates with local interfaces only which are essentially
> queues. The specific metaphor we use is that of an office worker with
> inboxes and outboxes with deliveries made between outboxes to inboxes.
> There also exists a simple environmental/service lookup facility which acts
> like an assistant in the above metaphor, and has natural similarities to a
> Linda type system.
> 
> (The actual rationale for the assistant facility though is based on
> biological systems. We have communicating linked concurrent components -
> which is much like many biological systems. However in addition to that
> most biological systems also have a hormonal system - which is part of the
> thinking behind the assistant system)
> 
> Generators (when embedded in a class) lend themselves very nicely to this
> sort of model in our experience /because/ they are limited to a single
> level (with regard to yield).
> 
> It's probably suitable for your students because we've tested the system on
> pre-university trainees, and vacation trainees, and found they're able to
> pick up the system, learn the basic ideas within a week or so (I have some
> exercises on how to build a mini- version if that helps), and build
> interesting systems. 
> 
> For example we had a pre-university trainee start with us at the beginning
> of the year, learn python, Kamaelia, and build a simple streaming system
> taking a video file, taking snapshots and sending those to mobile phones
> and PC's - this was over a period of 3 months. He'd only done a little bit
> of access in the past, and a little bit of VB. Well that as well as a
> simple learning system simulating a digital TV decode chain, but taking a
> script instead of a transport stream.
> 
> We recently made a 0.2.0 release of the system (not announced on c.l.p yet)
> that includes (basic) support for a wide range of multimedia/networked apps
> which might help people getting started. Some interesting new additions in
> the release are an IPython integration - allowing you to build Kamaelia
> systems on the fly using a shell, much like you can build unix pipelines,
> as well as a visual introspection tool (and network graph visualiser) which
> allows you to see inside systems as they are running. (This has turned out
> to be extremely useful - as you can expect with any CSP-type system)
> 
> The specific use cases you mention are also very closed aligned with our
> aims for the project. 
> 
> We're essentially working on making concurrency easy and natural to use,
> (starting from the domain of networked multimedia). You can do incremental
> development and transformation in exactly the way it sounds like you want,
> and build interesting systems. We're also working on the assumption that if
> you do that you can get performance later by specific optimisations (eg
> more CPUs).
>    * Example of incremental component development here:
>          http://tinyurl.com/dp8n7
> 
> By the time we reach a 1.0 release (of Kamaelia) we're also aiming to be
> able to integrate cleanly with Twisted (on Twisted's grounds), but it is
> highly usable already - especially in your area. (In CVS we have tools for
> building game type systems easily & Tk integration as well. Tk based CSP
> systems are particularly fun to work with (as in fun, not "fun" :). One
> project we are seriously looking at is a visual editor for these CSP-type
> systems, since that appears now to be low hanging fruit.
> 
> We've got a white paper about Kamaelia here:
>    * http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml
> 
> This is "textualisation" of a presentation I gave at ACCU earlier in the
> year and is an overview of the core areas of the system - hopefully enough
> to let you know whether to look further!
> 
> I also gave an updated talk at Europython - the presentation for which can
> be downloaded from here:
>    * http://www.python-in-business.org/ep2005/talk.chtml?talk=2589&track=692
> 
> Last week I also gave a more pragmatic, shorter talk at Open Tech which is
> an introduction to Kamaelia, it's goals, and several examples of CSP type
> systems ranging from simple audio clients/servers through to presentation
> tools. That presentation can be downloaded from here:
>    * http://kamaelia.sourceforge.net/Kamaelia_OT2005.tgz
> 
> Both the Europython & Open tech bundles actually include the core
> concurrency systems (called Axon), and the component modules (Kamaelia) in
> the bundle, in addition to the slides and install docs. The presentation
> tools included in the bundles are written in python using Kamaelia.
> 
> The project website is here: http://kamaelia.sourceforge.net/
> 
> 
>>I'm trying to develop distributed / Grid computing modules based on
>>python. The aim is to be able to use barriers for synchronisation and
>>channels for communication between processes running on a single box.
>>Then the jump to multiple processes on multiple boxes and eventually to
>>MPI implementations. Hopefully, each jump should not be that big a leap.
>>
>>Of course it would be nice if there was a robust way of managing
>>concurrency in python aswell ;-)
> 
> 
> It's getting there - the recent PEP regarding the ability to cause a
> generator to have an exception thrown inside it provides very similar to
> Unix signals as provided via "kill", which will be nice when 2.5 is
> released.
> 
> Hopefully you find the system useful,
> 
> Best Regards,
> 
> 
> Michael.
> --
> Michael Sparks, Senior R&D Engineer, Digital Media Group
> Michael.Sparks at rd.bbc.co.uk, http://kamaelia.sourceforge.net/
> British Broadcasting Corporation, Research and Development
> Kingswood Warren, Surrey KT20 6NP
> 
> This e-mail may contain personal views which are not the views of the BBC.
>