asynchronous events with multithreading and multiprocessing with circuits

James Mills prologic at shortcircuit.net.au
Thu Jan 8 06:45:18 CET 2009


Hi folks,

For those interested, I have just completed implementing
multiprocessing support for circuits (1). It has historically
always had multithreading support. These components
can be found in circuits.workers and are called:
Thread and Process

The reason these exist is to perform "work", ie: long-running
and potentially blocking tasks that should not be handled
with regular event handlers of components in a single
process or thread.

Here is the code that I used to test this with that utilizes
1 main process/thread, 1 additional thread and 1 additional process.

------------------------------------------------------------
#!/usr/bin/env python

from time import sleep

from circuits import listener, Event, Component, Debugger, Manager

from workers import Thread, Process

class Foo(Component):

    @listener("foo")
    def onFOO(self):
        print "foo"

    @listener("msg")
    def onMSG(self, s):
        print s

class A(Thread):

    def run(self):
        while self.running:
            try:
                sleep(2)
                self.push(Event("Hello from A"), "msg")
            except KeyboardInterrupt:
                self.running = False
                break

class B(Process):

    def run(self):
        while self.running:
            try:
                sleep(3)
                self.push(Event("Hello from B"), "msg")
            except KeyboardInterrupt:
                self.running = False
                break

manager = Manager()
#manager += Debugger()

foo = Foo()
a = A()
b = B()

manager += foo
manager += a
manager += b

a.start()
b.start()

while True:
    try:
        manager.flush()
        b.poll()
        sleep(1)
        manager.push(Event(), "foo")
    except KeyboardInterrupt:
        break

a.stop()
b.stop()

a.unregister()
b.unregister()
------------------------------------------------------------

I'll be later adapating primes.py found in the
examples/ directory of the circuits distribution
to utilize multiprocessing as well as distributed
processing. I may also provide other examples
of typical use-cases (distributed and parallel
csv processing for example).

The result of running the above test results in
the following (expected) output:

------------------------------------------------------------
$ python test.py
foo
foo
Hello from A
Hello from B
foo
foo
Hello from A
foo
Hello from B
foo
Hello from A
foo
foo
Hello from A
Hello from B
foo
^C
------------------------------------------------------------

Note: Each output line is approximately 1s in time.

Have a nice day and happy hacking :)

cheers
James

--
-- "Problems are solved by method"



More information about the Python-list mailing list