scraping with urllib2

[Javier Collado]

Hello,

To accept cookies, use the HTTPCookieProcessor as explained here:
http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/

Best regards,
    Javier

2010/1/27 Andre Engels <andreengels at gmail.com>:
> On Wed, Jan 27, 2010 at 6:26 AM, Patrick <whyadd9 at gmail.com> wrote:
>> I'm trying to scrape the attached link for the price listed $99.99:
>> http://bananarepublic.gap.com/browse/product.do?cid=41559&vid=1&pid=692392
>>
>> I can see the price if I view the source(I even turned off java and
>> javascript), but when I use urllib2, the price doesn't show up.
>>
>> Is there another library other than urllib2 that would work?
>
> To see that page you need to accept cookies from the site and send them back.
>
>
>
> --
> André Engels, andreengels at gmail.com
> --
> http://mail.python.org/mailman/listinfo/python-list
>