urlib - automatic cookie handling

[Pierre-Frédéric Caillaud]

	Yes.

	First fetch the URL which gives you the cookie. Parse the HTTP headers to  
get the cookie (use the header parsing function and get the Set-Cookie  
header). Then send the cookie in your headers along your next request, and  
you're in. For this you must use urllib2 which allows custom headers (use   
request.headers or something).


> I'm using urllib to post data to a web form by issuing a command similar  
> to this:
>
>
> filename, headers =  
> urllib.urlretrieve("http://www.thewebsitenamehere.com/servlet/com.blah.bloo.XmlFeed",  
> "content.txt", None, urllib.urlencode({"aParameter": "theValue"}))
>
> Now, the problem is that the above fails, since I am not sending a  
> session cookie. Visitors to the web sites' html submission form are sent  
> a session cookie which is given back to the server when they submit a  
> search via the browser, as often happens.
> Now, I could use urllib to get the form page and read the cookie from  
> the headers that are returned and then manually put that cookie in my  
> submission to the servlet, but my question is: is there a way to tell  
> urllib or some other part of HTTP handling in python that I want to  
> remember any cookie that is given to me, and give it back to that site  
> if I send requests later on?
>
> thanks
> alex
>