On Wed, 2008-06-18 at 04:13 -0400, kevin beckford wrote:

Is there any sort of job queue or similar , besides AsynQueue, which looked like exactly what i wanted, but is sadly undocumented, native to twisted? What is the twisted way to do such a thing? Is there something already available?

What kind of work are you queuing? If it's e.g. fetching a web page, you can just use do it using Twisted's scalable asynchronous APIs (maybe using a DeferredSemaphore if you want to limit number of conccurent queries.) If it's a blocking API you're calling (RDBMS queries are the typical example) then Twisted has a thread pool, as well as RDBMS-specific wrapper - see our documentation.

kevin beckford wrote:

Is there any sort of job queue or similar , besides AsynQueue, which looked like exactly what i wanted, but is sadly undocumented, native to twisted? What is the twisted way to do such a thing? Is there something already available?

Kevin, I have to confess that you're right about the "sadly undocumented" part. I keep promising myself (and others) that I will rectify the situation, but the motivation is just not as high as I'd like. I think that's true of a lot of free software; the author write the stuff to scratch his particular itch and then moves on. As the originator, he of course needs no documentation, and writing code is much more fun than writing words. The only thing I can offer in substitute at the moment are some really extensive docstrings in the code itself, and the usage examples embodied in the sAsync and AsynCluster packages, which itself are built on AsynQueue. Also, see a usage example for AsynCluster at http://foss.eepatents.com/trac/AsynCluster/wiki/Example_AsynCluster. Best regards, Ed