urllib and login with passwords

Rob Williscroft rtw at freenet.co.uk
Sat Jul 26 21:43:09 CEST 2008

Jive Dadson wrote in news:SvJik.317138$fz6.173226 at fe08.news.easynews.com in 

> Hey folks!
> There are various web pages that I would like to read using urllib, but 
> they require login with passwords. Can anyone tell me how to find out 
> how to do that, both in general and specifically for YouTube.com.

A typical pattern is submit a form to login and get a cookie back,
subsuquent request with the cookie set are then "loged in".

import cookielib, urllib2

cj = cookielib.CookieJar()

opener = urllib2.build_opener( urllib2.HTTPCookieProcessor(cj) )

page = opener.open( LOGIN_URL, data = LOGIN_FORM )

page = opener.open( DOWNLOAD_URL )

print page.read() 

You will need to work out what goes in LOGIN_FORM, it likely
something like:

LOGIN_FORM = "username=name&password=pass&submit-button=Some+value"

where username, password and submit-button are the name of the 
controls on the form you would normally login from.

If the form has an enctype='multipart/form-data' then things
get a little more complex, possibly start here:



More information about the Python-list mailing list