Question on using urllib

Peter Otten __peter__ at web.de
Thu Nov 11 18:37:27 CET 2010


Olivier Scalbert wrote:

> Sorry if this question is not 100% python related, it is also related to
> http or html ...
> 
> Everything takes place in this page:
> http://www.infometeo.be/klanten/KYC/campage2mega.php
> 
> With the following program, I can download the 2 small meteo images,
> without any problem:
> 
> import urllib
> 
> 
urllib.urlretrieve("http://www.infometeo.be/wwweather/currentimage.php?icao=EBFN",
> "meteo1.jpg")
> urllib.urlretrieve("http://www.meteokust.be/metstatgr.php?s=zb",
> "meteo2.jpg")
> 
> But I would like to download the main background picture.
> 
> By looking inside the html page, it seems it's url is:
> http://www.infometeo.be/img.php?iid=1057

> I try to download it with python:
> urllib.urlretrieve("http://www.infometeo.be/img.php?iid=1057",
> "picture.jpg")

The problem is indeed not Python-related. The provider of the images doesn't 
like what you're trying to do and verifies the referer, i. e. that the page 
you claim to be coming from is acceptable. Here's one way to satisfy that 
check:

>>> from urllib2 import Request, urlopen
>>> r = Request("http://www.infometeo.be/img.php?iid=1057", 
headers=dict(Referer="http://www.infometeo.be/img.php?iid=1057"))
>>> img = urlopen(r).read()
>>> with open("tmp.jpg", "w") as f: f.write(img)
...
>>>

Now if you are using that workaround a lot he may come up with more 
sophisticated techniques. So dont ;)

Peter



More information about the Python-list mailing list