Web client login with redirection and cookies

[John]

Hi,

I'm trying to login to an intranet site that uses cookies and
redirection for a web scraping script.  Are there any good examples of
how to accomplish this in Python?  I recently managed to get this type
of login working in Perl, and am now playing around with this in
Python.

I have only been working with Perl for 6 months casually for a few
database and web scraping applications for automating reporting, and
have been thinking about switching to Python before I invest too much
more time.  I am already struggling a bit in maintaining my fairly
small amount of code as I only work on it a few days out of a month
and thought Python might benefit me in this regard.

As far as I can tell, it looks like I need to use urllib2 and
ClientCookie, but so far I haven't gotten anything to work.  Can
anyone suggest an approach based on the working Perl version?

use LWP::UserAgent;
use HTTP::Cookies;
use HTTP::Request::Common;
my $cookie_jar = HTTP::Cookies::Netscape->new(
			file => "e:/cookies.txt",
			autosave => 1
);
my $browser = LWP::UserAgent->new;
$browser->cookie_jar( $cookie_jar );
$browser->agent('Mozilla/4.76 [en] (Windows NT 4.0; U)');
push @{$browser->requests_redirectable}, 'POST';
my $response = $browser->post(
		'http://site/login_gen.php',
				[
				'loginid' => 'id',
				'password' => 'pass',
				'next' => 'http://site/reports/drill.php',
				'submit' => 'ENTER'
				]
);

I would appreciate any advice.

Thanks,
John