waiting for html to load: a followup

John J. Lee jjl at pobox.com
Thu Aug 26 21:06:40 CEST 2004

Josh <joshl at commenspace.org> writes:
> Anyway, I decided to forget IE and I am now trying to use urllib2 to
> open up the page, read it, etc.  My problem is the page has a built-in
> refresh and I don't know how to have python re-read the page until
> it's ready to hand over the links.

ClientCookie does that (HTTPRefreshProcessor and HTTPEquivProcessor in


I recommend using the alpha release.  The interface will change a
little soon, but you almost certainly won't notice.

> An example of the page is:
> http://edcw2ks23.cr.usgs.gov/Website/zipship/waiting.jsp?areaList=49.0,47.0,-122.0,-124.08&prodList=NED,
> I believe I need to read the header, grab the cookie session id, and
> add it back to the header.  I can do all thus, but I'm stuck on

It'll do the cookies too :-)

> probably very simple syntax to re-read the page rather than open a new
> connection, if that makes sense (I'm new to http as well as python).

You don't need to ensure it's the same connection.  In fact, you can't
easily do that with urllib2 (or ClientCookie) as it is currently.



More information about the Python-list mailing list