Infinite loops and synchronization

[Piet van Oostrum]

>>>>> Vincent Gulinao <vincent.gulinao at gmail.com> (VG) wrote:

>VG> lst = list()
>VG> (lst populated by async twisted deferred callbacks)

>VG> while True:
>VG> 	if len(lst) == SOME_NUMBER:
>VG> 		return lst

>VG> Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
>VG> Q2: operating on list from threads (mostly appends) must be safe,
>VG> right (synchronization)?

I am not familiar enough with twisted, but I think the principle is
independent from twisted.

This loop will not hog memory but it will hog CPU time.

You should use a synchronisation construct like threading.Condition or a
Semaphore. Here is my suggestion with a Condition:

Global somewhere:
       lst_cond = Condition()

In your loop:

lst = list()  # Why not lst = []?

while True: # strange while/if combo
	if len(lst) == SOME_NUMBER:
		return lst

Make that:

with lst_cond:
    while len(lst) < SOME_NUMBER:
        lst_cond.wait()
    return lst

In the callback:

with lst_cond:
    lst.append(new_value)
    lst_cond.notify()

In case you don't have a python that supports the with statement (not
even `from future') you should use:

lst_cond.acquire()
try:
    .....
finally:
    lst_cond.release()

I think the solution with a semaphore is less elegant.

global: sem = Semaphore()

loop:
for i in range(SOME_NUMBER): 
    sem.acquire()
return lst

In callback:

lst.append(new_value)
sem.release()

*Be careful: I haven't tested this code (not even syntax checked). So
consider it pseudo code.*
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org