Web Crawler - Python or Perl?

subeen tamim.shahriar at gmail.com
Mon Jun 9 20:21:54 CEST 2008


On Jun 10, 12:15 am, Stefan Behnel <stefan... at behnel.de> wrote:
> subeen wrote:
> > can use urllib2 module and/or beautiful soup for developing crawler
>
> Not if you care about a) speed and/or b) memory efficiency.
>
> http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/
>
> Stefan

ya, beautiful soup is slower. so it's better to use urllib2 for
fetching data and regular expressions for parsing data.


regards,
Subeen.
http://love-python.blogspot.com/



More information about the Python-list mailing list