Message: 3 Date: Mon, 22 Jan 2007 00:32:16 +0000 From: Phil Mayers <p.mayers@imperial.ac.uk> Subject: Re: [Twisted-Python] More Questions about Custom Reactors To: Twisted general discussion <twisted-python@twistedmatrix.com> Message-ID: <45B40610.7020607@imperial.ac.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed I am sorry that I am tardy in responding. I was trying out your suggestions. Right now, my main question is under which conditions do deferred fail to fire? How do I know that the reactor is still running? Would Manhole help? (Unfortunately I haven't been able to get gtk to work Manhole with Twisted 2.4)

Perhaps you could explain what you're actually trying to achieve, rather than the solution you have decided on? An indeed, why you want to use stackless at all - Twisted is predicated around the assumption of a stackful i.e. single-threaded Python.

I understand your questioning. The main reason I am using Stackless Python is for its process execution state pickling abilities. A second reason is that I find Stackless's communication model very handy for modeling the type of thread synchronisation I wish to do. I know that Twisted assumes a stackful Python and it is an uphill battle. However the gains from getting Twisted to work with Stackless are so great, I think it is worthwhile. For the most part, I have succeeded in having Stackless Python work with Twisted. If you read the Stackless mailing list, I have posted examples. Soon, I will post on the Wiki. My problems stem from situations where I have two or more concurrent tasklets (Stackless threads) that do the following) tasklet[0] - waiting for a Twisted Request Handler tasklet[1] - making a client.getPage call tasklet[2] - making a client.getPage call tasklet[3] - some other operation tasklet[n] - some other operation under these conditions, if tasklet[0] is executed first, the other tasklets will block indefinitely. My problems occur from trying to get around this. Originally I tried taskLoop calling stackless.schedule() to give other tasklets a chance to execute but this would cause reactor problems. Hence the current approach of looking at getting Twisted to run in its own thread.

Alternatively you might be wanting long-running jobs to be interleaved with other code transparently but without using real threads?

I am not sure how to do this.

If you're proposing using queues to communicate between a Twisted thread running the reactor and a stackless thread running your micro-threads, I don't see why you need to modify Twisted at all.

I am a newbie to Twisted. Based on what I am reading, I current assumption is that I would need to

You can just run the stackless microthreads from one "real" thread (started via callInThread), pass jobs from Twisted to the "real" thread and probably a stackless uthread, and pass the replies back to the twisted thread with a callFromThread.

So, can you not just do this?:

Philip, first the good news : thanks a million! This is the exactly the type of example I wanted. It works. The bad news - under certain conditions I am losing deferreds. I think there are deadlock issues but that is not Twisted's fault per se. Unfortunately debuggers work poorly with Stackless so it is difficult for me to know what is happening. I tried telneting but my server is not responding - so I guess that implies that Twisted is still listening but the reactor is not working. Again, any suggestions would be more than welcome. Cheers, Andrew ____________________________________________________________________________________ Bored stiff? Loosen up... Download and play hundreds of games for free on Yahoo! Games. http://games.yahoo.com/games/front