drt@un.bewaff.net (Doobee R. Tzeck) writes:

I'm trying to build a non interactive client app with twisted. Let's assume it is a webcrawler. I'm not sure how to do that. Might be like this:

Create a Loop Object which checks if the desired number of connections are running and if not create new ones an add them as connectors.

Or should I rip out application.run() and write my own look to better handle the starting of new crawlers?

You may want to look at twisted/test/interactive/ldap_parallelsearch.py, which is an LDAP client that does 5 simultaneous connections, 10 asynchronous queries (results flow in arbitrary order) per connection. Of course, it ain't HTTP. -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}

drt@un.bewaff.net (Doobee R. Tzeck) writes:

I'm trying to build a non interactive client app with twisted. Let's assume it is a webcrawler. I'm not sure how to do that. Might be like this:

Create a Loop Object which checks if the desired number of connections are running and if not create new ones an add them as connectors.

Or should I rip out application.run() and write my own look to better handle the starting of new crawlers?

You may want to look at twisted/test/interactive/ldap_parallelsearch.py, which is an LDAP client that does 5 simultaneous connections, 10 asynchronous queries (results flow in arbitrary order) per connection. Of course, it ain't HTTP. -- tv@{{hq.yok.utu,havoc,gaeshido}.fi,{debian,wanderer}.org,stonesoft.com} double a,b=4,c;main(){for(;++a<2e6;c-=(b=-b)/a++);printf("%f\n",c);}