Making a persistent HTTP connection

Alan Kennedy alanmk at
Mon Nov 14 22:54:19 CET 2005

[David Rasmussen]
>> I use urllib2 to do some simple HTTP communication with a web server. 
>> In one "session", I do maybe 10-15 requests. It seems that urllib2 
>> opens op a connection every time I do a request. Can I somehow make it 
>> use _one_ persistent connection where I can do multiple GET->"receive 
>> data" passes before the connection is closed?

[Diez B. Roggisch]
> Are you sure HTTP supports that? 

Yes, HTTP 1.1 definitely supports multiple requests on the same connection.

Some HTTP 1.0 clients supported persistent connections through the use 
of the non-standard "keep-alive" header.

> And even if it works - what is the problem with connections being created?

The URL above describes the benefits of persistent connections. The 
primary problem of the old style of one-request-per-connection is the 
creation of more sockets than are necessary.

To the OP: neither urllib nor urllib2 implements persistent connections, 
but httplib does. See the httplib documentation page for an example.

However, even httplib is "synchronous", in that it cannot pipeline 
requests: the response to the first request must be competely read 
before a second request can be issued.


alan kennedy
email alan:    

More information about the Python-list mailing list