Running Python on SMP machine

[Erno Kuusela]

>>>>> "William" == William Park <parkw at better.net> writes:

 | Hi Erno,  can you elaborate on this?

 | My program is not particularly I/O intensive -- just slurping single
 | table at the beginning, and writing a smaller table at the end.  But,
 | within my program, there are a lot of sections where calculations are
 | inherently independent.  Eg.
 |     c = a+b	<- section 1, independent of section 2
 |     d = a-b	<- section 2, independent of section 1
 |     e = c/d	<- using result of the two sections.

the speedup you will get will depend on the length of those sections.
if they are long, you will benefit. you can use for example sockets
or pipes to transmit results between processes. so, you have to
try to break your task into as large independent sections as possible.

if you are on a unix variant of some kind, you can use fork() and that
makes things much esier.

something like...

    a = 5
    b = 3
    r, w = os.pipe()
    pid = os.fork()
    if pid == 0:
        result = a - b
        cPickle.dump(result, os.fdopen(w, 'w'))
        os._exit(0)
    c = a + b
    d = cPickle.load(os.fdopen(r, 'r'))
    os.wait()

on windows i guess you could start a bunch of
copies and then send tasks for them from a "main" process.

  -- erno