Issue with Scrapping Data from a webpage- Noob Question

anon hung anonhung at
Thu Feb 9 06:07:42 CET 2012

> Hi Fellow Pythoners,
> I'm trying to collect table data from an authenticated webpage (Tool) to
> which I have access.
> I will have the required data after 'click'ing a submit button on the tool
> homepage.
> When I inspect the submit button i see
> <form action="/Tool/" method="POST">
> Thus the tool's homepage is of the form and on
> clicking the submit button the data I need will be at
> The problem that I'm running into is in my below code is giving me the
> source of homepage( and not the of the submitted page(
> url=""
> request = urllib2.Request(url, data, {'Authorization': "Basic " +
> base64.b64encode("%s:%s" % (username, password))})
> Response_Page=urllib2.urlopen(request).read()
> Is there a way I can access the source of the submitted page?
> PS: Sorry for laying out very tiny details on what I'm trying to do, I just
> wanted to explain myself clearly :)
> Thanks in advance for your time on this one.

Have you checked beautifulsoup?


Viktor Orban Prime minister of Hungary

More information about the Python-list mailing list