[New-bugs-announce] [issue12133] ResourceWarning in urllib.request

[Ezio Melotti]

New submission from Ezio Melotti <ezio.melotti at gmail.com>:

In case of error (e.g. timeout error), urllib.request leaves the socket open:

import urllib.request as ur
import socket
s = socket.socket()
s.bind(('localhost', 10000))
s.listen(0)
socket.setdefaulttimeout(5)
ur.urlopen('http://localhost.localdomain:10000')

outputs:

sys:1: ResourceWarning: unclosed <socket.socket object, fd=4, family=2, type=1, proto=6>

Traceback (most recent call last):
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 1146, in do_open
    r = h.getresponse()  # an HTTPResponse instance
  File "/home/wolf/dev/py/py3k/Lib/http/client.py", line 1046, in getresponse
    response.begin()
  File "/home/wolf/dev/py/py3k/Lib/http/client.py", line 346, in begin
    version, status, reason = self._read_status()
  File "/home/wolf/dev/py/py3k/Lib/http/client.py", line 308, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/wolf/dev/py/py3k/Lib/socket.py", line 279, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 138, in urlopen
    return opener.open(url, data, timeout)
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 369, in open
    response = self._open(req, data)
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 387, in _open
    '_open', req)
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 347, in _call_chain
    result = func(*args)
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 1163, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 1148, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error timed out>
>>>

AFAIU, when urlopen returns or raises, the socket can be closed, so the attached patch adds a "finally" that calls close() on the HTTPConnection object.  The test suite pass (except for a mock that was missing the close method), but I'm not 100% sure that it's always safe to call close().

This ResourceWarning has been exposed by test_packaging.

----------
components: Library (Lib)
files: issue12133.diff
keywords: needs review, patch
messages: 136433
nosy: davide.rizzo, ezio.melotti, jhylton, pitrou
priority: normal
severity: normal
stage: patch review
status: open
title: ResourceWarning in urllib.request
type: resource usage
versions: Python 3.3
Added file: http://bugs.python.org/file22047/issue12133.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12133>
_______________________________________