<p>Hello:</p>
<p>Mechanize is another good module for automating this kind of thing. </p>
<p>HTH,</p>
<p>Aaron</p>
<div class="gmail_quote">On Jun 25, 2011 1:43 PM, "Stephen McInerney" <<a href="mailto:spmcinerney@hotmail.com">spmcinerney@hotmail.com</a>> wrote:<br type="attribution">> <br>> <br>> What do people use for scraping on a website requiring (login form-based) authentication?<br>
> BeautifulSoup: does not handle authentication or cookiesScrapy: does but more heavyweight paradigm to learn, incl. XPath<br>> Some discussion: <a href="http://stackoverflow.com/questions/4328271/best-way-for-a-beginner-to-learn-screen-scraping-with-python">http://stackoverflow.com/questions/4328271/best-way-for-a-beginner-to-learn-screen-scraping-with-python</a><br>
> <br>> Thanks,<br>> Stephen<br>> <br>> <br></div>