[Medusa-dev] Re: [Quixote-users] Poor upload (POST) performance... [PATCH]

[Ken Sugino]

== Related issue: Poor GET performace for large files:

1) http_request.push uses "simple_producer" which copies string each time
"more" is called. Instead "scanning_producer" should be used:

--- medusa-0.5.4/http_server.py	2003-12-06 01:34:27.000000000 -0500
+++ medusa-0.5.4.mod1/http_server.py	2004-02-16 18:46:52.000000000 -0500
@@ -157,8 +157,8 @@
      def push (self, thing):
          if type(thing) == type(''):
-            self.outgoing.append(producers.simple_producer (thing))
+            self.outgoing.append(producers.scanning_producer (thing))
+            # 2003.04.29 KS changed from simple producer
          else:
              self.outgoing.append(thing)


2) Sometimes (e.g. from script_handler in thread/thread_handler.py),
"http_channel.push" is called directly which is actually inherited from
"asynchat.async_chat.push" which uses again "simple_producer".
"http_channel" should probably override this "push" to use 
"scanning_producer":

@@ -536,6 +539,7 @@
+	def push (self, data):
+         # 20031202 KS override this async_chat method to use 
scanning_producer
+         self.producer_fifo.push (producers.scanning_producer(data))
+         self.initiate_send()



== Threading problem on windows xp (os.name=='nt')
"asyncore.dispatcher.listen" restricts number of connection to one if 
requested
connection is more than 5.
|	def listen(self, num):
|        self.accepting = 1
|        if os.name == 'nt' and num > 5:
|            num = 1
|        return self.socket.listen(num)

"http_server.__init__" calls self.listen(1024) which on windows ends up
with only one connection and affect threaded connections. Ideally, 
asyncore.py
should be modified but following change is OK as well.

@@ -562,8 +565,14 @@
          self.bind ((ip, port))

          # lower this to 5 if your OS complains
-        self.listen (1024)
-
+        #self.listen (1024)
+        #KS
+        if os.name == 'nt':
+            self.listen(5)
+        else:
+            self.listen(1024)
+        #/KS
+
          host, port = self.socket.getsockname()
          if not ip:
              self.log_info('Computing default hostname', 'warning')


== Other bugs related to thread_handler.py
1) thread/thread_handler.py line 165 'put', 'post' -> 'PUT', 'POST'
since crack_request in http_server.py dropped .lower()

2) thread/thread_handler.py line 214 lines=lines[:-1] should be
commented out for binary data transfer.

--- medusa-0.5.4/thread/thread_handler.py	2003-12-06 01:36:23.000000000 
-0500
+++ medusa-0.5.4.mod1/thread/thread_handler.py	2004-02-16 
18:46:52.000000000 -0500
@@ -161,7 +161,8 @@
          except KeyError:
              pass

-        if request.command in ('put', 'post'):
+        if request.command in ('PUT', 'POST', 'OPTIONS'): # KS, OPTIONS: 
explorer? webdav?
              # PUT data requires a correct Content-Length: header
              # (though I bet with http/1.1 we can expect chunked encoding)
              request.collector = collector (self, request, env)
@@ -209,7 +210,10 @@
              self.buffer = self.buffer + data
              lines = string.split (self.buffer, '\n')
              # ignore the last piece, it is either empty, or a partial line
-            lines = lines[:-1]
+
+            #lines = lines[:-1]
+	    # (KS) need last line if you are sending binary data
+
              # look for something un-header-like
              for i in range(len(lines)):
                  li = lines[i]


I've been using medusa with these modifications for a couple of months now 
w/o
problem. (http://egret.bio.brandeis.edu/nikki/NiKKi)

Ciao,
Ken