Re : [Twisted-Python] Multiple thread pools
Hi,
I can see now how 'untwisted' my threads concerns are, please forget my newbie ununderstanding ;-)
Still, I need to ensure that I won't have more than x similar calls at a time.
Is there a way, maybe using some of the deferred classes (deferredqueue,..)
to ensure that there is no more than x simultaneous similar calls (wrapped in deferreds), the others being pending,
until one is finished. My question might be stupid, but I'm quite new to twisted and the way to implement
this behaviour is not clear to me (even if I found great help in the o'reilly book and the docs).
Thanks,
Stéphane
----- Message d'origine ----
De : Stéphane Brault
On Thu, 11 May 2006 15:32:08 +0000 (GMT), Stéphane Brault
Hi, I can see now how 'untwisted' my threads concerns are, please forget my newbie ununderstanding ;-) Still, I need to ensure that I won't have more than x similar calls at a time. Is there a way, maybe using some of the deferred classes (deferredqueue,..) to ensure that there is no more than x simultaneous similar calls (wrapped in deferreds), the others being pending, until one is finished
I think twisted.internet.defer.DeferredSemaphore is what you're looking for. There is an example (someone else's example) of using it to do something similar here: http://pastebin.com/702986 Moe
Still, I need to ensure that I won't have more than x similar calls at a time. Is there a way, maybe using some of the deferred classes (deferredqueue,..) to ensure that there is no more than x simultaneous similar calls (wrapped in deferreds), the others being pending, until one is finished.
You are on the right track. A couple possible approaches are in this mailing list thread: http://twistedmatrix.com/pipermail/twisted-python/2006-January/012388.html -- Nicola Larosa - http://www.tekNico.net/ Much as everyone thinks they want financial security, the happiest people are not those who have it, but those who like what they do. -- Paul Graham, January 2006
Thanks Moe,
the example seems to work, which is nice ;-)
If I understand well DeferredSemaphore(x) guarantees that no more than x concurent calls that acquired it
will be run together. Am I wrong when inferring that DeferredList is not compulsory ? And that I adding calls to the semaphore
as I need them will ensure the behaviour I want (as my tasks come, acquire the semaphore, then release it) ?
Thaks again,
Stéphane
----- Message d'origine ----
De : Moe Aboulkheir
Hi, I can see now how 'untwisted' my threads concerns are, please forget my newbie ununderstanding ;-) Still, I need to ensure that I won't have more than x similar calls at a time. Is there a way, maybe using some of the deferred classes (deferredqueue,..) to ensure that there is no more than x simultaneous similar calls (wrapped in deferreds), the others being pending, until one is finished
I think twisted.internet.defer.DeferredSemaphore is what you're looking for. There is an example (someone else's example) of using it to do something similar here: http://pastebin.com/702986 Moe _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Thanks Nicola,
I'm investigating this further.
Stéphane
----- Message d'origine ----
De : Nicola Larosa
Still, I need to ensure that I won't have more than x similar calls at a time. Is there a way, maybe using some of the deferred classes (deferredqueue,..) to ensure that there is no more than x simultaneous similar calls (wrapped in deferreds), the others being pending, until one is finished.
You are on the right track. A couple possible approaches are in this mailing list thread: http://twistedmatrix.com/pipermail/twisted-python/2006-January/012388.html -- Nicola Larosa - http://www.tekNico.net/ Much as everyone thinks they want financial security, the happiest people are not those who have it, but those who like what they do. -- Paul Graham, January 2006 _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Thu, 11 May 2006 17:43:05 +0000 (GMT), Stéphane Brault
Am I wrong when inferring that DeferredList is not compulsory ?
It depends whether or not you want to know when all of the tasks have completed, which is what the example uses it for. Moe
Thanks Moe,
that's what I inferred, I don't need to know when all tasks have completed. I only need to run tasks as they arrive,
whether scheduled or called by a user and guarantee that no more than a certain number are running simultineaously.
Stéphane
----- Message d'origine ----
De : Moe Aboulkheir
Am I wrong when inferring that DeferredList is not compulsory ?
It depends whether or not you want to know when all of the tasks have completed, which is what the example uses it for. Moe _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (3)
-
Moe Aboulkheir -
Nicola Larosa -
Stéphane Brault