check if an URL exists without opening it

[Jim Meier]

On Fri, 09 Aug 2002 09:45:29 -0600, Michael Chermside wrote:

>> I would like to check if an URL exists.
>> (for instance http://www.yahoo.com/try.pdf)
>> The method urllib.open is unsatisfactory because the URL (which will be a
>> file in my program) is opened ! So it can take too long time, just to check
>> the existence !
> 
> Unfortunately, this is impossible for many kinds of URLs. http: urls, 
> for instance, can only be detected by an attempt to download them.

This is wrong. HTTP supports the HEAD request, which is just like GET
except it only returns information headers. Getting 404 back means there's
no document there; getting a content-length means there is.

I don't use urllib, but I'm sure someone will point you at how to do it
with the standard library.

-Jim