how to handle network failures

[Aahz]

In article <7bee5155-f1a1-44aa-90f6-eaabb361729f at v6g2000prd.googlegroups.com>,
harryos  <oswald.harry at gmail.com> wrote:
>
>class DataGrabber(threading.Thread):
>    def __init__(self,url):
>        threading.Thread.__init__(self)
>        self.url=url
>    def run(self):
>        data=self.get_page_data()
>        process_data(data)
>
>    def get_page_data():
>        try:
>            f=urllib.urlopen(self.url)
>            data=f.read(1024)
>        except IOError:
>            #wait for some time and try again
>            time.sleep(120)
>            data=self.get_page_data()
>        return data

Use urllib2 so that you can set a timeout (Python 2.6+).
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair