I recently posted a "webchecker" script which is not quite a web grabber. It is also on ftp.python.org in /pub/tmp (until I find a better place for it). It should be straightforward to turn this into a grabber.
A similar but even easier solution is (and I did it :) : Use Guido's /tools/scripts/ftpmirror.py on the appropriate directory of ftp.python.org, since the whole site is accessible through FTP. If you just want to grab HTML and use a fresh directory it works immediately. There is a minor NT/W95 related bug with binary files and the rename function. I'd just let it grab the whole ftp://ftp.python.org/pub/www.python.org/doc/
Much longer ago I posted a "robot" thingie for the same purpose that is better forgotten. I believe I saw a post from someone else who had turned it into a grabber, recently.
But ftpmirror is still alive, isn't it? Sorry, I was nearly ready with the last patches for the NT problems, firewall traversal and that stuff, but by testing it I wrote it over with the old version (yes, like a greenhorn), so it make take some time until I'll want to write it again :-/ BTW, to the makers of FTPlib: It is not fire proof. There are circumstances where it will hang, or report a wrong error. I tested it with a WinGate beta which gave incomplete messages partially. Someone should enhance the error handling. (in the hope you don't suggest me..:) - chris _______________ DOC-SIG - SIG for the Python Documentation Project send messages to: doc-sig@python.org administrivia to: doc-sig-request@python.org _______________