Web queries in Python

Mon Sep 4 16:15:03 EDT 2006

Joseph wrote:
> Hi,
> Excel uses web queries to read data (like financial/weather data) from
> web. How to do the same in python? Even pointers to such would be of
> help.
> 
sholden at bigboy ~/hwb
$ python
Python 2.5b2 (trunk:50713, Jul 19 2006, 16:04:09)
[GCC 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
Started with C:/Steve/.pythonrc
  >>> import urllib
  >>> f = urllib.urlopen("http://www.google.com/")
  >>> data = f.read()
  >>> print data
<html><head><meta http-equiv="content-type" content="text/html; 
charset=ISO-8859
-1"><title>Google</title><style><!--
body,td,a,p,.h{font-family:arial,sans-serif}
.h{font-size:20px}
.q{color:#00c}
--></style>
[...]
<a
  href="/intl/en/ads/">Advertising Programmes</a> - <a 
href=/services/>Business Solutions</a> - <a 
href=/intl/en/about.html>About Google</a> - <a 
href=http://www.google.com/ncr>Go to Google.com</a></font><p><font 
size=-2>©2006 Google</font></p></center></body></html>
  >>>

So that's an easy way to get hold of the raw data. Look at htmllib or 
BeautifulSoup to parse it intot he structures you want.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC/Ltd          http://www.holdenweb.com
Skype: holdenweb       http://holdenweb.blogspot.com
Recent Ramblings     http://del.icio.us/steve.holden