Not necessarily related to python Web Crawlers

subeen tamim.shahriar at
Sun Jul 6 12:32:48 CEST 2008

On Jul 5, 2:31 pm, disappeare... at wrote:
> Hi
> Does anyone here have a good recommendation for an open source crawler
> that I could get my hands on? It doesn't have to be python based. I am
> interested in learning how crawling works. I think python based
> crawlers will ensure a high degree of flexibility but at the same time
> I am also torn between looking for open source crawlers in python vs C
> ++ because the latter is much more efficient(or so I heard. I will be
> crawling on very cheap hardware.)
> I am definitely open to suggestions.
> Thx

You can check my python blog. There are some tips and codes on


More information about the Python-list mailing list