client-server parallellised number crunching
Hans Georg Schaathun
hg at schaathun.net
Wed Apr 27 20:28:06 CEST 2011
On Thu, 28 Apr 2011 00:58:22 +1000, Chris Angelico
<rosuav at gmail.com> wrote:
: thousand threads? a couple of million? In Python, it'll probably end
: up pretty similar; chances are you won't be taking much advantage of
: multiple CPUs/cores (because the threads will all be waiting for
: socket read, or the single thread will mostly be waiting in select()),
: so it's mainly a resource usage issue. Probably worth testing with
: your actual code.
For my own application, the performance issue is rather negligible.
I don't have more than about 50 idle CPU-s which I can access easily,
and even if I had, it would always stop at 6-7000 functions calls
to evaluate. In the current test runs using 4 clients and one master
on a quad-core, the master never uses more than around 7% of a core,
and that includes some simple post-processing as well as com's.
In short, my philosophy is that it works, so why change it?
But, I am aware that some more technical adept programmers think
otherwise, and I am quite happy with that :-)
:-- Hans Georg
More information about the Python-list