client-server parallellised number crunching

Hans Georg Schaathun hg at schaathun.net
Wed Apr 27 20:28:06 CEST 2011


On Thu, 28 Apr 2011 00:58:22 +1000, Chris Angelico
  <rosuav at gmail.com> wrote:
:  thousand threads? a couple of million? In Python, it'll probably end
:  up pretty similar; chances are you won't be taking much advantage of
:  multiple CPUs/cores (because the threads will all be waiting for
:  socket read, or the single thread will mostly be waiting in select()),
:  so it's mainly a resource usage issue. Probably worth testing with
:  your actual code.

For my own application, the performance issue is rather negligible.
I don't have more than about 50 idle CPU-s which I can access easily,
and even if I had, it would always stop at 6-7000 functions calls
to evaluate.  In the current test runs using 4 clients and one master
on a quad-core, the master never uses more than around 7% of a core,
and that includes some simple post-processing as well as com's. 

In short, my philosophy is that it works, so why change it?

But, I am aware that some more technical adept programmers think
otherwise, and I am quite happy with that :-)

-- 
:-- Hans Georg



More information about the Python-list mailing list