urllib2 - closing sockets

Steve Holden sholden at holdenweb.com
Fri Jun 4 15:08:21 CEST 2004


Anand Pillai wrote:
> I recently noted that urllib2.urlopen(...) for http:// urls
> does not make an explicit call to close the underlying 
> HTTPConnection socket once the data from the socket is read.
> 
> This might not be required since the garbage collector will
> close & collect open sockets that are not closed, but it might
> cause the system to run out of socket memory if there are 
> multiple threads, each opening a socket and the gc not running
> in between.
> 
> This specifically happens in my HarvestMan program which uses
> multiple threads to achieve fast offline web downloads.
> 
> A patch to fix this in urllib2.py would be nice.
> 
> Thanks
> 
> -Anand

In which case you'd be well advised to add this as a bug report on 
Sourceforge, as that is the only way to guarantee it will come to (and 
stay in) the developers' attention.

It isn't that hard to do.

regards
  Steve



More information about the Python-list mailing list