[Tutor] can I walk or glob a website?

[Alan Gauld]

"Dave Angel" <davea at ieee.org> wrote

>> "Albert-Jan Roskam" <fomcl at yahoo.com> wrote
>>> How can I walk (as in os.walk) or glob a website?
>>
>> I don't think there is a way to do that via the web.

> It has to be (more or less) possible.  That's what google does for 
> their search engine.

Google trawls the site following links. If thats all he wants then its 
fairly easy.
I took it he wanted to actually trawl the server getting *all* the pdf 
files not
just the published pdfs...

Depends what the real requirement is.

Alan G.