Question on using urllib

[Peter Otten]

Olivier Scalbert wrote:

> Sorry if this question is not 100% python related, it is also related to
> http or html ...
> 
> Everything takes place in this page:
> http://www.infometeo.be/klanten/KYC/campage2mega.php
> 
> With the following program, I can download the 2 small meteo images,
> without any problem:
> 
> import urllib
> 
> 
urllib.urlretrieve("http://www.infometeo.be/wwweather/currentimage.php?icao=EBFN",
> "meteo1.jpg")
> urllib.urlretrieve("http://www.meteokust.be/metstatgr.php?s=zb",
> "meteo2.jpg")
> 
> But I would like to download the main background picture.
> 
> By looking inside the html page, it seems it's url is:
> http://www.infometeo.be/img.php?iid=1057

> I try to download it with python:
> urllib.urlretrieve("http://www.infometeo.be/img.php?iid=1057",
> "picture.jpg")

The problem is indeed not Python-related. The provider of the images doesn't 
like what you're trying to do and verifies the referer, i. e. that the page 
you claim to be coming from is acceptable. Here's one way to satisfy that 
check:

>>> from urllib2 import Request, urlopen
>>> r = Request("http://www.infometeo.be/img.php?iid=1057", 
headers=dict(Referer="http://www.infometeo.be/img.php?iid=1057"))
>>> img = urlopen(r).read()
>>> with open("tmp.jpg", "w") as f: f.write(img)
...
>>>

Now if you are using that workaround a lot he may come up with more 
sophisticated techniques. So dont ;)

Peter