Async Client with 1K connections?

Aahz aahz at pythoncraft.com
Wed Feb 11 20:00:55 CET 2004


In article <zeWdnXYoFaHsSbTd4p2dnA at comcast.com>,
William Chang <williamichang at hotmail.com> wrote:
>
>One of the intended uses is indeed a next-gen web spider.  I did the
>math, and yes I will need about 10 cutting-edge PCs to spider like
>you-know-who.  

Note that while you-know-who makes extensive use of Python, I don't
think they're using it for spidering/searching.  I do have some
background writing a spider in Python, using Verity's engine for
indexing/retrieval, but we were using threading rather than
asyncore-style operations.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Argue for your limitations, and sure enough they're yours."  --Richard Bach



More information about the Python-list mailing list