[BangPypers] HarvestMan with eventlets

Sridhar Ratnakumar sridhar.ratna at gmail.com
Thu Dec 27 08:02:13 CET 2007


On Dec 27, 2007 11:46 AM, Anand Balachandran Pillai <abpillai at gmail.com> wrote:
> Hi,
>
> I have been planning to rewrite HarvestMan using a non-blocking approach
> rather than the current design of achieving concurrency using multiple
> threads (which do not scale very well in Python).

+1

> Also the fact that twisted
> requires an explicit event loop call is not very good and makes it difficult
> to write an application which has its own separate event loop.

You can write a custom reactor. For an example, take a look at the
gtk2 reactor, http://twistedmatrix.com/trac/browser/trunk/twisted/internet/gtk2reactor.py#L146

> Eventlets look promising because it is implemented using coroutines without
> an explicit event loop call, instead the event loop is run transparently when
> required. Also due to its design, it can be easily merged with Stackless, which
> I think it is already on the way to.

I came across this stackless example for Twisted,
http://www.imperialviolet.org/binary/st.py  I don't know how mature is
Twisted's stackless support.

> I have been evaluating twisted for sometime for doing this, but I have not
> been able to wrap my brain around twisted.

.. around *yet*. Speaking from my experience, it is only a short term
struggle - once you become a little comfortable with the feel it,
Twisted programming becomes a joy in itself. When I was learning
Twisted, the knowledge of its component system helped a lot,
http://twistedmatrix.com/projects/core/documentation/howto/components.html

I also noticed that Eventlet seems to support Twisted, although it is
currently broken:

> twistedsupport.py : Support for using eventlet with twisted. See http://twistedmatrix.com.
> Currently broken.

-- 
http://nearfar.org/


More information about the BangPypers mailing list