Web-crawling
Rene Pijlman
reply.in.the.newsgroup at my.address.is.invalid
Sat Oct 4 08:22:53 EDT 2003
John Bradbury:
>I am trying to develop a special putpose crawler using htmllib & urllib.
>How do you tell the server application that you are a modern browser and can
>handle frames?
I don't know of any "I can handle frames" header and I don't see why the
server would care, but you could mimic the User-agent header sent by a
modern browser.
--
René Pijlman
More information about the Python-list
mailing list