os.system and wget

John Hunter jdhunter at nitace.bsd.uchicago.edu
Wed Nov 13 11:39:55 EST 2002


>>>>> "zunbeltz" == zunbeltz  <zunlatex at hotmail.com> writes:

    zunbeltz> os.system('wget -q -O foo.txt http://foo.html')

You may want to look at the popen commands:
http://python.org/doc/current/lib/os-newstreams.html#os-newstreams.

For your example:

  import os
  h = os.popen('wget -q -O foo1.txt http://foo.html')
  h.close()
  s = open('foo1.txt').read()

Also, for the specific case of wget, python's builtin urllib can
provide a bunch of these services

  import urllib
  h = urllib.urlretrieve('http://foo.html', 'foo2.txt')

Or if you want to work with the returned data directly in python, you
can use urlopen, which returns a filehandle

  h = urllib.urlopen('http://foo.html')
  for line in h.readlines():
     print line

You can get fancier (cookies, recursive retrieval).  See websucker as
an example python script built around urllib that has much of the wget
functionality: http://www.pythonware.com/people/fredrik/websucker.htm.
If you have the python src, you'll find it in the src tree at
Tools/webchecker/websucker.py

John Hunter



More information about the Python-list mailing list