[Tutor] Help error 504

[Peter Otten]

Gonzalo V wrote:

> i am new in python and its very intuitive! but i am in problems with that
> code.

As Danny already explained an exception can only be caught if the function 
is inside a try ... except ...:

try:
    req = urllib.request.urlopen(...)
except urllib.error.HTTPError as e:
    ...    

> how can simulate or emulate an error 504?

(1) You can set up a server under your own control and temporarily use that:

$ cat serve_error.py
import bottle

@bottle.route("/<status:int>")
def set_status(status):
    bottle.response.status = status

if __name__ == "__main__":
    bottle.run(host="localhost", port=8000)
$ python3 serve_error.py 
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://localhost:8000/
Hit Ctrl-C to quit.

In another shell:

>>> import urllib.request
>>> urllib.request.urlopen("http://localhost:8000/504")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in 
http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 504: Gateway Timeout

The server is flexible enough to produce other errors like the well-known 
404:

>>> try: urllib.request.urlopen("http://localhost:8000/404")
... except Exception as e: print(e)
... 
HTTP Error 404: Not Found

> how can simulate or emulate an error 504?

(2) Doing this from within your own script is called "mocking". While Laura 
provided a link that probably explains how to do this right and how unit 
tests help avoiding that the same or similar problems will resurface in the 
future I'll provide you with the dead simple core of mocking:

You have a function that sometimes shows unwanted behaviour? Temporarily 
replace it with another function that always shows this behaviour -- until 
the surrounding code has learnt to deal with it. 

For example:

> i wrote this code and it cant handle 504 error:
> import urllib.request
> import urllib.error
> from bs4 import BeautifulSoup
> import re, csv
> from FuncionCsv import LlenarCsv


def raise_httperror_504(url):
    raise urllib.error.HTTPError(
        url, 504, 
        "I'm afraid I can't do this", None, None)

urllib.request.urlopen = raise_httperror_504

> fhand=open('isbn.txt')
> #csvfile=open('ResultadoScrapping.csv', 'w', newline='')
> for line in fhand:
>     req=urllib.request.urlopen('XXXXXXXX'+line)
>     resp=req.read()
>     soup=BeautifulSoup(resp,'html.parser')
>     try:
>         origen=soup.find(string=re.compile("Origen:
> ")).find_next().get_text()
>         nombre=soup.find(name="h1",itemprop="name").get_text()
>         precioAhora=soup.find(name="p",class_="precioAhora").get_text()
>         d=soup.find(name="p",class_="stock").get_text()
>         disp=d.split()
>     except AttributeError:
>         disp="no encontrado"
>         nombre=''
>         origen=''
>         precioAhora=''
>     except urllib.error.HTTPError as e:
>             if e.getcode()==504:
>                 disp = "sin respuesta del servidor"
>                 print (e.getcode(),disp)
>                 csvfile.close()
> 
>     print(line,nombre,origen,precioAhora,disp)
>     line1=line.split()
>     LlenarCsv('Resultado.csv',line1,nombre,origen,precioAhora,disp)