Python Web Site Copier?

Jay Parlar jparlar at
Sun Aug 12 05:28:34 CEST 2001

Not sure if it's *exactly* what you want, but one of the code samples in  Chun's "Core Python Programming" is a web 
crawler, that "takes a starting Web address, downloads that page and all other pages whose links appear in succeeding 
pages, but only those which are in the same domain as the starting page". 

You might be looking for something more advanced, but you can at least give it a look. Wesley Chun's page is and this code in particular can be found at

Good luck to you!

> Has anybody written a Python version of a utility like sitecopy?  (For
> synchronizing a web site on a server with a copy maintained on one's hard
> drive.)  From my search, I see that a few months ago somebody was working on a
> Python wrapper for sitecopy, but I would prefer a pure-Python solution for a
> variety of reasons (which I won't go into).
> I am up to the job of writing my own but of course I don't want to if I don't
> have to!
> Thanks in advance.  My apologies if this isn't completely on topic.
> -- 
> ======================================================================
> Paul Sidorsky                                          Calgary, Canada
> paulsid at            

Jay Parlar
Software Engineering III
McMaster University
Hamilton, Ontario, Canada

"Though there are many paths
At the foot of the mountain
All those who reach the top
See the same moon."

More information about the Python-list mailing list