How do I correctly download Wikipedia pages?

Steven D'Aprano steven at
Thu Nov 26 04:45:19 CET 2009

I'm trying to scrape a Wikipedia page from Python. Following instructions 

I use the URL "" instead 
of just "". But instead of getting the 
page I expect, and can see in my browser, I get an error page:

>>> import urllib
>>> url = ""
>>> print urllib.urlopen(url).read()
Our servers are currently experiencing a technical problem. This is 
probably temporary and should be fixed soon

(Output is obviously truncated for your sanity and mine.)

Is there a trick to downloading from Wikipedia with urllib?


More information about the Python-list mailing list