[New-bugs-announce] [issue18144] FD leak in urllib2

[Claudio Freire]

New submission from Claudio Freire:

While other issues already exist about this problem, this particular case is unlike other issues, and I didn't think it a good idea to merge with those.

Under some very specific circumstances (sending a POST request with more data than an unknown threshold), at least one socket remains unclosed after calling close() on urllib2.urlopen's returned file object.

While I marked the only versions I could confirm exhibit the issue, I believe this is an issue on all versions.

This started in pypy[0], although it applies to CPython as well (albeit the reference counting GC is less likely to delay closing of the FD as much as in pypy).

I'm attaching the same server used to trigger this issue in pypy, works the same with CPython.

To trigger the leak, open an interpreter and do this (copypaste from pypy, CPython does not cause the leak because decref immediately closes the leak, but it will issue a wraning if ran with -Wall). See pypy's issue tracker[0] for detilas.

>>>> import os, urllib2
>>>> req = """{"imp": [{"h": 50, "battr": ["9", "10", "12"], "api": 3, "w": 320,
"instl": 0, "impid": "5d6dedf3-17bb-11e2-b5c0-1040f38b83e0"}]""" * 10
>>>> r = urllib2.Request("http://localhost:8000/bogus?src=1", req)
>>>> u = urllib2.urlopen(r)
>>>> v = u.read()
>>>> u.close()
>>>> os.system("ls -alh /proc/%d/fd/*" % os.getpid())
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/0 -> /dev/pts/5
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/1 -> /dev/pts/5
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/2 -> /dev/pts/5
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/3 ->
socket:[2086998]
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/5 -> /dev/pts/5
lrwx------ 1 claudiofreire users 64 Jun  4 15:08 /proc/26203/fd/6 -> /dev/pts/5
0
>>>> 


[0] https://bugs.pypy.org/issue867

----------
components: Library (Lib)
files: bogus.py
messages: 190687
nosy: Claudio.Freire
priority: normal
severity: normal
status: open
title: FD leak in urllib2
type: behavior
versions: Python 2.7, Python 3.1, Python 3.2
Added file: http://bugs.python.org/file30477/bogus.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18144>
_______________________________________