[Python-ideas] solving multi-core Python

Wes Turner wes.turner at gmail.com
Thu Jun 25 23:51:48 CEST 2015

On Thu, Jun 25, 2015 at 10:25 AM, Sturla Molden <sturla.molden at gmail.com>

> On 25/06/15 16:31, Nick Coghlan wrote:
>  3. The potential for collisions between objects means it isn't an
>> embarrassingly parallel problem where the different computational
>> threads can entirely ignore the existence of the other threads
> Well, you can have a loop that updates all particles, e.g. by calling a
> coroutine associated with each particle, and then this loop is an
> embarrassingly parallel problem. You don't need to associate each particle
> with its own thread.
> It is bad to teach students to use one thread per particle anyway.
> Suddenly they write a system that have thousands of threads.

Understood that this is merely an example re: threading, but
BSP seems to be the higher-level algorithm for iterative graphs with

* https://en.wikipedia.org/wiki/Bulk_synchronous_parallel
* https://giraph.apache.org/
* https://spark.apache.org/docs/latest/
  * https://spark.apache.org/docs/latest/api/python/ (no graphx BSP yet,
* https://github.com/xslogic/phoebus (Erlang, HDFS, Thrift)
* https://github.com/mnielsen/Pregel/blob/master/pregel.py (Python)

Intra-machine optimization could also be useful.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150625/74036ab1/attachment-0001.html>

More information about the Python-ideas mailing list