[Tutor] open a webpage which may be unavailable

[Alan Gauld]

"pileux systeme" <nogentstanford at yahoo.fr> wrote

>  I am trying to retrieve data from several webpages. 
> My problem is the following: after a random number of requests, 
> the page I'm trying to open is unavailable (and I get an IOError). 
> Note that the page may become available if I try again 

This is similar to the problem faced by marketeers doing 
outbound phone calls. The customer may not be home so 
you have to try again later (maybe several times).

The normal solution to this is to buoild a list of failed addresses 
and then after processing the original list go back and 
process the exceptions list. Repeat for N iterations or 
until exceptions list is empty.

This approach saves multiple attempts in clise proximitry 
to a faulty address when you could be processing productive 
addresses so is time efficient.

The only snag is if you have to process the addresses in 
a given sequence. In that case you may need to create 
a list of groups and if any address within a group fails 
put the entire group in the exceptions list.

Coincidentally, given my recent message on Mainframe 
technologies, this is the same technique used in high 
volume data processing (on mainframes) where millions 
of orders or bills etc need to be processed quickly. Rather than 
delay the multitude to fix errors on a case by case basis 
a quick pass is made collecting faulty records and a 
second pass (maybe with a more rigorouds algorithm)
is made of the exceptions.

HTH,

-- 
Alan Gauld
Author of the Learn to Program web site
http://www.freenetpages.co.uk/hp/alan.gauld