HTML Parsing

Stefan Behnel stefan_ml at
Sun Jun 29 07:26:05 CEST 2008

disappearedng at wrote:
> I am trying to build my own web crawler for an experiement and I don't
> know how to access HTTP protocol with python.
> Also, Are there any Opensource Parsing engine for HTML documents
> available in Python too? That would be great.

Try lxml.html. It parses broken HTML, supports HTTP, is much faster than
BeautifulSoup and threadable, all of which should be helpful for your crawler.


More information about the Python-list mailing list