urllib and persistence

[Mark Carter]

> > to know your surname, for example. So I don't think that wget will
> > help.
> 
> Actually wget is able to load cookies from Netscape-format cookies.txt
> files, so if you did a first login with Netscape you could then use
> wget with the --load-cookies option.  I read on wget's manual, but
> haven't tried myself, that from Internet Explorer you can produce a
> textfile that wget can use

It works!

I used wget rather than python to do the download - it just seems
plain easier to do it that way.

Here's a summary of what I discovered, in case others
find it useful:

One can view cookie details in Nestcape (7.02) by selecting
menu item Edit - Preferences
category Privacy & Security
subsection Cookies
button Manage Stored cookies
tab Stored Cookies

This presents a list of cookies whose details you can inspect.

Cookies can be exported from Internet Explorer 
menu File - Import & Export

This exports the cookies as a text file.

TIP:
You may need to edit some of the entries in the cookies.txt file.
Most of the entries are of the form
www.boo.com
and probably require no ammendement, but where you see
boo.com
you may have to prepend a dot, thus:
.boo.com
This can happen if there is more than one way to access the server,
e.g. via www1.boo.com and www2.boo.com




Alex Martelli <aleax at aleax.it> wrote in message news:<7t_fa.10826$i26.247708 at news2.tin.it>...
> Mark Carter wrote:
> 
> >> >   http://wwwsearch.sourceforge.net/ClientCookie/ may thus
> >> > prove of help.
> >> 
> >> As will wget, if you want a non-python solution.
> > 
> > Alas, the site requires logging in, but not in a strict
> > account/password way - it wants
> > to know your surname, for example. So I don't think that wget will
> > help.
> 
> Actually wget is able to load cookies from Netscape-format cookies.txt
> files, so if you did a first login with Netscape you could then use
> wget with the --load-cookies option.  I read on wget's manual, but
> haven't tried myself, that from Internet Explorer you can produce a
> textfile that wget can use:
> 
> """
> You can produce a cookie file Wget can use by using the File menu, Import 
> and Export, Export Cookies. This has been tested with Internet Explorer 5; 
> it is not guaranteed to work with earlier versions. 
> """
> 
> There are a few recipes in the Python Cookbook for dealing with
> cookies as saved by both Netscape and IE.
> 
> 
> > I must admit that I'm a bit lacking in understanding how urllib2
> > and/or ClientCookie
> > are supposed to work. Does ClientCookie work with Internet Explorer
> > cookies, or not?
> 
> MSIECookies SHOULD let you use cookies saved by IE -- I think it's
> prominently enough explained among the FAQ's at
> http://wwwsearch.sourceforge.net/ClientCookie/ .  Using netscape-
> formatted cookies may be preferable even there, though -- better
> tested/clearer/more understood (as it's text, it's easier to
> understand and "reverse engineer".
> 
> 
> 
> Alex