[Python-checkins] python/dist/src/Lib httplib.py,1.59,1.60

[jhylton@users.sourceforge.net]

Update of /cvsroot/python/python/dist/src/Lib
In directory usw-pr-cvs1:/tmp/cvs-serv23177

Modified Files:
	httplib.py 
Log Message:
Send HTTP requests with a single send() call instead of many.

The implementation now stores all the lines of the request in a buffer
and makes a single send() call when the request is finished,
specifically when endheaders() is called.

This appears to improve performance.  The old code called send() for
each line.  The sends are all short, so they caused bad interactions
with the Nagle algorithm and delayed acknowledgements.  In simple
tests, the second packet was delayed by 100s of ms.  The second send was
delayed by the Nagle algorithm, waiting for the ack.  The delayed ack
strategy delays the ack in hopes of piggybacking it on a data packet,
but the server won't send any data until it receives the complete
request.

This change minimizes the problem that Nagle + delayed ack will cause
a problem, although a request large enough to be broken into two
packets will still suffer some delay.  Luckily the MSS is large enough
to accomodate most single packets.

XXX Bug fix candidate?


Index: httplib.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/httplib.py,v
retrieving revision 1.59
retrieving revision 1.60
diff -C2 -d -r1.59 -r1.60
*** httplib.py	12 Jul 2002 14:04:08 -0000	1.59
--- httplib.py	16 Jul 2002 21:21:11 -0000	1.60
***************
*** 480,483 ****
--- 480,484 ----
      def __init__(self, host, port=None, strict=None):
          self.sock = None
+         self._buffer = []
          self.__response = None
          self.__state = _CS_IDLE
***************
*** 544,548 ****
                  raise NotConnected()
  
!         # send the data to the server. if we get a broken pipe, then close
          # the socket. we want to reconnect when somebody tries to send again.
          #
--- 545,549 ----
                  raise NotConnected()
  
!         # send the data to the server. if we get a broken pipe, then closesdwu
          # the socket. we want to reconnect when somebody tries to send again.
          #
***************
*** 558,561 ****
--- 559,579 ----
              raise
  
+     def _output(self, s):
+         """Add a line of output to the current request buffer.
+         
+         Aassumes that the line does *not* end with \r\n.
+         """
+         self._buffer.append(s)
+ 
+     def _send_output(self):
+         """Send the currently buffered request and clear the buffer.
+ 
+         Appends an extra \r\n to the buffer.
+         """
+         self._buffer.extend(("", ""))
+         msg = "\r\n".join(self._buffer)
+         del self._buffer[:]
+         self.send(msg)
+ 
      def putrequest(self, method, url, skip_host=0):
          """Send a request to the server.
***************
*** 566,569 ****
--- 584,588 ----
  
          # check if a prior response has been completed
+         # XXX What if it hasn't?
          if self.__response and self.__response.isclosed():
              self.__response = None
***************
*** 595,608 ****
          if not url:
              url = '/'
!         str = '%s %s %s\r\n' % (method, url, self._http_vsn_str)
  
!         try:
!             self.send(str)
!         except socket.error, v:
!             # trap 'Broken pipe' if we're allowed to automatically reconnect
!             if v[0] != 32 or not self.auto_open:
!                 raise
!             # try one more time (the socket was closed; this will reopen)
!             self.send(str)
  
          if self._http_vsn == 11:
--- 614,620 ----
          if not url:
              url = '/'
!         str = '%s %s %s' % (method, url, self._http_vsn_str)
  
!         self._output(str)
  
          if self._http_vsn == 11:
***************
*** 665,670 ****
              raise CannotSendHeader()
  
!         str = '%s: %s\r\n' % (header, value)
!         self.send(str)
  
      def endheaders(self):
--- 677,682 ----
              raise CannotSendHeader()
  
!         str = '%s: %s' % (header, value)
!         self._output(str)
  
      def endheaders(self):
***************
*** 676,680 ****
              raise CannotSendHeader()
  
!         self.send('\r\n')
  
      def request(self, method, url, body=None, headers={}):
--- 688,692 ----
              raise CannotSendHeader()
  
!         self._send_output()
  
      def request(self, method, url, body=None, headers={}):
***************
*** 1203,1206 ****
--- 1215,1219 ----
              print "https://%s%s" % (host, selector)
              hs = HTTPS()
+             hs.set_debuglevel(dl)
              hs.connect(host)
              hs.putrequest('GET', selector)
***************
*** 1214,1219 ****
                  for header in headers.headers: print header.strip()
              print
- 
-     return
  
      # Test a buggy server -- returns garbled status line.
--- 1227,1230 ----