[Python Edinburgh] Functional Programming in Python

[Thomas Figg]

On 24 Dec 2010, at 10:32, Mark Smith wrote:

> To fully utilise these processors, then, we need a new approach to coding that can distribute the processing load 

You might want to look at chapel - a language from cray - to see a modern attempt at a language for distributed
and parallel number crunching. Alternatively the recent language 'Plaid' has some novel ways of using data-flow for parallelism.

Anyway, there are two main approaches to multiple processors: parallel processing and concurrency. What works for one might not work for the other. 

For example, google use map-reduce (a batch framework for out of core processing) and pregel (using a bulk synchronous parallel model for graph calculations) - both with elements of parallelism but massively different architectures.

Parallelising an algorithm is hard, and the sufficiently smart compilers haven't arrived yet - at least not for imperative languages. Functional programs are easier to re-order and rewrite because of the lack of side effects/referential transparency. Languages with restrictions on aliasing (fortran) and assignment (single assigment c) are simpler to optimise. Using annotations is bug prone, and debugging parallel programs is no mean feat. 

Even when you do parallelise something, often the returns aren't that great - the overhead of synchronisation
eats away at performance gains (c.f ahmdals law)

We also face the larger problem of locality. We still program as if access to memory is universally cheap, when in reality some memory is cheaper to read than other bits (see cache hits).  This gets much harder to deal with
when you bring in extra cores and machines.

I guess what I'm trying to say is that effective use of multiple processors requires good design from the outset, rather than annotations on for loops. 

Aside: You might enjoy this talk about the architecture behind a low latency trade system, and why the avoided
the traditional models http://bit.ly/i22X6C


-tef