fiber(cooperative multi-threading)

[Arnaud Delobelle]

On Dec 22, 12:10 pm, Akihiro KAYAMA <kay... at st.rim.or.jp> wrote:
> Hi all.
>
> I found cooperative multi-threading(only one thread runs at once,
> explicit thread switching) is useful for writing some simulators.
> With it, I'm able to be free from annoying mutual exclusion, and make
> results deterministic.
>
> For this purpose, and inspired by Ruby(1.9) fiber, I wrote my own
> version of fiber in Python.
>
> It just works, but using native Python threads for non-preemptive
> threading is not cost-effective. Python has generator instead but it
> seemed to be very restricted for general scripting. I wish I could
> write nested (generator) functions easily at least.
>
> Is there any plan of implementing real (lightweight) fiber in Python?
>
> ------------------------------------------------------------
> import threading
>
> class Fiber(threading.Thread):
>     def __init__(self):
>         threading.Thread.__init__(self)
>
>         self.semaphore_running = threading.Semaphore(0)
>         self.semaphore_finish = None
>         self.val = None
>
>         self.setDaemon(True)
>         self.start()
>         self.start = self.start_fiber
>
>     def start_fiber(self):
>         self.semaphore_finish = threading.Semaphore(0)
>         self.semaphore_running.release()
>         self.semaphore_finish.acquire()
>
>     def run(self):              # override
>         self.semaphore_running.acquire()
>         self.main()
>         if self.semaphore_finish is not None:
>             self.semaphore_finish.release()
>
>     def switchto(self, fiber, val=None):
>         fiber.val = val
>         fiber.semaphore_running.release()
>         self.semaphore_running.acquire()
>         return self.val
>
>     def main(self):             # should be overridden
>         pass
>
> class F1(Fiber):
>     def main(self):
>         print "f1 start"
>         self.switchto(f2)
>         print "f1 foo"
>         v = self.switchto(f2)
>         print "f1 v=%s world" % v
>         self.switchto(f2, "OK")
>         print "f1 end"
>
> class F2(Fiber):
>     def main(self):
>         print "f2 start"
>         self.switchto(f1)
>         print "f2 bar"
>         result = self.switchto(f1, "Hello, ")
>         print "f2 result=%s" % result
>         print "f2 end"
>         self.switchto(f1)
>
> f1 = F1()
> f2 = F2()
>
> print "start"
> f1.start()
> print "end"
>
> -- kayama

I am not really familiar with ruby but these fibers seem to be some
sort of coroutines.  Since python 2.5, generators can be sent values,
this can be used to implement what you want.  I have had a got at it
for fun and this is what I came up with: note that there is no need
for a specific class, or for threads.  I have tested this only on your
example, but it gives the same result as your code :)

-------------------------------------

# A 'fiber' is a generator function that yields tuples (fiber, value)
# or (fiber,).  The 'run' function starts the the fiber and carries on
# with the fiber it yields.  If the fiber also yields a value, then
# this value is sent to the 'next' fiber.

def run(fiber):
   it, val = fiber(), []
   fibers = { fiber: it }
   try:
       while True:
           n = it.send(*val) if val else it.next()
           co, val = n[0], n[1:]
           it = fibers.get(co)
           if it is None:
               fibers[co] = it = co()
   except StopIteration:
       return val[0] if val else None

# Now the two 'fibers'

def f1():
    print "f1 start"
    yield f2,
    print "f1 foo"
    v = yield f2,
    print "f1 v=%s world" % v
    yield f2, "OK"
    print "f1 end"

def f2():
    print "f2 start"
    yield f1,
    print "f2 bar"
    result = yield f1, "Hello, "
    print "f2 result=%s" % result
    print "f2 end"
    yield f1,

print "start"
run(f1)
print "end"

--------------------------------------

HTH

--
Arnaud