Hello,
Here's a demo HTTP server that returns 10 MB of random data each time a client connects.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True def render_GET(self, request): return data
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Now, when I run N clients simultaneously from a different host I see that the server's memory consumption increases by N*10 MB. I can't reproduce this example when running the clients from the same host as the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \ --rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the test it jumps to 117.2 MB. My questions are then:
1. Given that 'data' is a global variable, eventually read-only as well, why is it replicated for each request? And who is replicating it?
2, What would be the proper way to re-write this example so that there is one and only one 'data' structure at any time?
Thank you,
On 05:19 am, psanchez@fosstel.com wrote:
Hello,
Here's a demo HTTP server that returns 10 MB of random data each time a client connects.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True def render_GET(self, request): return data
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Now, when I run N clients simultaneously from a different host I see that the server's memory consumption increases by N*10 MB. I can't reproduce this example when running the clients from the same host as the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \ --rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the test it jumps to 117.2 MB. My questions are then:
- Given that 'data' is a global variable, eventually read-only as
well, why is it replicated for each request? And who is replicating it?
It's copied as part of the process of writing it to the socket. You can't write 10MB at once, and you can't slice a string (to throw away the part that you did manage to write) without making a copy of part of it.
2, What would be the proper way to re-write this example so that there is one and only one 'data' structure at any time?
Split data up into ~32kB-64kB chunks and write them to the request individually. Then each chunk can just be dropped with no copying.
Jean-Paul
On 10-12-21 09:00 AM, exarkun@twistedmatrix.com wrote:
On 05:19 am, psanchez@fosstel.com wrote:
Hello,
Here's a demo HTTP server that returns 10 MB of random data each time a client connects.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True def render_GET(self, request): return data
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Now, when I run N clients simultaneously from a different host I see that the server's memory consumption increases by N*10 MB. I can't reproduce this example when running the clients from the same host as the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \ --rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the test it jumps to 117.2 MB. My questions are then:
- Given that 'data' is a global variable, eventually read-only as
well, why is it replicated for each request? And who is replicating it?
It's copied as part of the process of writing it to the socket. You can't write 10MB at once, and you can't slice a string (to throw away the part that you did manage to write) without making a copy of part of it.
2, What would be the proper way to re-write this example so that there is one and only one 'data' structure at any time?
Split data up into ~32kB-64kB chunks and write them to the request individually. Then each chunk can just be dropped with no copying.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Thanks Jean-Paul,
Here is my modified example, unfortunately with the same bad results regarding memory consumption.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True
def render_GET(self, request): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): request.write(chunk) s = s + CHUNK_SIZE
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Shall I add some delay before writing each chunk?
On 02:34 pm, psanchez@fosstel.com wrote:
On 10-12-21 09:00 AM, exarkun@twistedmatrix.com wrote:
On 05:19 am, psanchez@fosstel.com wrote:
Hello,
Here's a demo HTTP server that returns 10 MB of random data each time a client connects.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True def render_GET(self, request): return data
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Now, when I run N clients simultaneously from a different host I see that the server's memory consumption increases by N*10 MB. I can't reproduce this example when running the clients from the same host as the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \ --rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the test it jumps to 117.2 MB. My questions are then:
- Given that 'data' is a global variable, eventually read-only as
well, why is it replicated for each request? And who is replicating it?
It's copied as part of the process of writing it to the socket. You can't write 10MB at once, and you can't slice a string (to throw away the part that you did manage to write) without making a copy of part of it.
2, What would be the proper way to re-write this example so that there is one and only one 'data' structure at any time?
Split data up into ~32kB-64kB chunks and write them to the request individually. Then each chunk can just be dropped with no copying.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Thanks Jean-Paul,
Here is my modified example, unfortunately with the same bad results regarding memory consumption.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True
def render_GET(self, request): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): request.write(chunk) s = s + CHUNK_SIZE
You've just moved the copying-by-slicing out of the transport and into your Resource. :) You need to do all that copying/slicing at the beginning, where it only needs to happen once, so that every render can share those allocated strings.
Jean-Paul
On 10-12-21 10:59 AM, exarkun@twistedmatrix.com wrote:
On 02:34 pm, psanchez@fosstel.com wrote:
On 10-12-21 09:00 AM, exarkun@twistedmatrix.com wrote:
On 05:19 am, psanchez@fosstel.com wrote:
Hello,
Here's a demo HTTP server that returns 10 MB of random data each time a client connects.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True def render_GET(self, request): return data
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Now, when I run N clients simultaneously from a different host I see that the server's memory consumption increases by N*10 MB. I can't reproduce this example when running the clients from the same host as the server; the test goes so fast that I can't gather any useful data.
I run the test using the following httperf command on a different host and looking at the Gnome system monitor in the server (top will do as well).
httperf --server 192.168.1.10 --port 8880 --uri /test \ --rate 10 --num-conn 10
When the server is idle memory consumption is 17.1 MB, but during the test it jumps to 117.2 MB. My questions are then:
- Given that 'data' is a global variable, eventually read-only as
well, why is it replicated for each request? And who is replicating it?
It's copied as part of the process of writing it to the socket. You can't write 10MB at once, and you can't slice a string (to throw away the part that you did manage to write) without making a copy of part of it.
2, What would be the proper way to re-write this example so that there is one and only one 'data' structure at any time?
Split data up into ~32kB-64kB chunks and write them to the request individually. Then each chunk can just be dropped with no copying.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Thanks Jean-Paul,
Here is my modified example, unfortunately with the same bad results regarding memory consumption.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True
def render_GET(self, request): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): request.write(chunk) s = s + CHUNK_SIZEYou've just moved the copying-by-slicing out of the transport and into your Resource. :) You need to do all that copying/slicing at the beginning, where it only needs to happen once, so that every render can share those allocated strings.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)
make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
I tried also preparing the chunks in a TestPage.__init__() implementation. Same results. So, where exactly do I have to put the make_chunks() steps?
Thanks,
On 04:43 pm, psanchez@fosstel.com wrote:
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
This version has flat memory usage on my system - 33MB all the way through. Jean-Paul
On 10-12-21 12:47 PM, exarkun@twistedmatrix.com wrote:
On 04:43 pm, psanchez@fosstel.com wrote:
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
This version has flat memory usage on my system - 33MB all the way through. Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Jean-Paul,
Did you try this running the clients on a different machine than the one running the server? I find that if I run both the server and the clients (using httperf) on the same machine things behave differently. I can only see the memory consumption problem consistently when running the clients on a separate machine (which would be my real use case).
BTW, I'm running these test on Ubuntu 10.04, Python 2.6.5, twisted 10.0.0-2ubuntu2.
On 07:02 pm, psanchez@fosstel.com wrote:
On 10-12-21 12:47 PM, exarkun@twistedmatrix.com wrote:
On 04:43 pm, psanchez@fosstel.com wrote:
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
This version has flat memory usage on my system - 33MB all the way through. Jean-Paul
Did you try this running the clients on a different machine than the one running the server? I find that if I run both the server and the clients (using httperf) on the same machine things behave differently. I can only see the memory consumption problem consistently when running the clients on a separate machine (which would be my real use case).
BTW, I'm running these test on Ubuntu 10.04, Python 2.6.5, twisted 10.0.0-2ubuntu2.
Argh. I just ran it over loopback, so I wasn't accurately testing the case you described. James Knight just reminded me that the transport will still find a way to copy data in this case, so you're right that this still isn't a working solution. Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
Request implements IConsumer, which means you can pass an IProducer provider to its registerProducer method. When the send buffer gets close to empty, the IProducer's resumeProducing method will be called and you can write some more bytes to the Request.
There are some more complications and subtleties in the IProducer/IConsumer interfaces, which you can read about at http://twistedmatrix.com/documents/current/core/howto/producers.html.
Basically, you'll end up with something like:
from twisted.python.log import err from twisted.protocols.basic import FileSender from twisted.web.server import NOT_DONE_YET
def render_GET(self, request): producer = FileSender() d = producer.beginFileTransfer(StringIO(chunk), request) def finished(ignored): request.finish() d.addErrback(err, "Streaming data to client failed") d.addCallback(finished) return NOT_DONE_YET
Sorry for the earlier mis-information.
Jean-Paul
On 10-12-21 02:24 PM, exarkun@twistedmatrix.com wrote:
On 07:02 pm, psanchez@fosstel.com wrote:
On 10-12-21 12:47 PM, exarkun@twistedmatrix.com wrote:
On 04:43 pm, psanchez@fosstel.com wrote:
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
This version has flat memory usage on my system - 33MB all the way through. Jean-Paul
Did you try this running the clients on a different machine than the one running the server? I find that if I run both the server and the clients (using httperf) on the same machine things behave differently. I can only see the memory consumption problem consistently when running the clients on a separate machine (which would be my real use case).
BTW, I'm running these test on Ubuntu 10.04, Python 2.6.5, twisted 10.0.0-2ubuntu2.
Argh. I just ran it over loopback, so I wasn't accurately testing the case you described. James Knight just reminded me that the transport will still find a way to copy data in this case, so you're right that this still isn't a working solution. Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
Request implements IConsumer, which means you can pass an IProducer provider to its registerProducer method. When the send buffer gets close to empty, the IProducer's resumeProducing method will be called and you can write some more bytes to the Request.
There are some more complications and subtleties in the IProducer/IConsumer interfaces, which you can read about at http://twistedmatrix.com/documents/current/core/howto/producers.html.
Basically, you'll end up with something like:
from twisted.python.log import err from twisted.protocols.basic import FileSender from twisted.web.server import NOT_DONE_YET def render_GET(self, request): producer = FileSender() d = producer.beginFileTransfer(StringIO(chunk), request) def finished(ignored): request.finish() d.addErrback(err, "Streaming data to client failed") d.addCallback(finished) return NOT_DONE_YETSorry for the earlier mis-information.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Thanks again Jean-Paul,
This one will take me some time to digest and prototype. I'll give it a whirl to try to implement a solution and I'll post back my results.
On 10-12-21 02:53 PM, Pedro I. Sanchez wrote:
On 10-12-21 02:24 PM, exarkun@twistedmatrix.com wrote:
On 07:02 pm, psanchez@fosstel.com wrote:
On 10-12-21 12:47 PM, exarkun@twistedmatrix.com wrote:
On 04:43 pm, psanchez@fosstel.com wrote:
OK, I guess I'm being slow :-( Here's another version, same results.
import os from twisted.internet import reactor from twisted.web.server import Site from twisted.web.resource import Resource
CHUNK_SIZE = 32*1024 data = os.urandom(10*1024*1024) chunks = []
def make_chunks(): s = 0 for chunk in iter(lambda: data[s:s+CHUNK_SIZE], ''): chunks.append(chunk) s = s + CHUNK_SIZE
class TestPage(Resource): isLeaf = True
def render_GET(self, request): for chunk in chunks: request.write(chunk)make_chunks() root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
This version has flat memory usage on my system - 33MB all the way through. Jean-Paul
Did you try this running the clients on a different machine than the one running the server? I find that if I run both the server and the clients (using httperf) on the same machine things behave differently. I can only see the memory consumption problem consistently when running the clients on a separate machine (which would be my real use case).
BTW, I'm running these test on Ubuntu 10.04, Python 2.6.5, twisted 10.0.0-2ubuntu2.
Argh. I just ran it over loopback, so I wasn't accurately testing the case you described. James Knight just reminded me that the transport will still find a way to copy data in this case, so you're right that this still isn't a working solution. Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
Request implements IConsumer, which means you can pass an IProducer provider to its registerProducer method. When the send buffer gets close to empty, the IProducer's resumeProducing method will be called and you can write some more bytes to the Request.
There are some more complications and subtleties in the IProducer/IConsumer interfaces, which you can read about at http://twistedmatrix.com/documents/current/core/howto/producers.html.
Basically, you'll end up with something like:
from twisted.python.log import err from twisted.protocols.basic import FileSender from twisted.web.server import NOT_DONE_YET def render_GET(self, request): producer = FileSender() d = producer.beginFileTransfer(StringIO(chunk), request) def finished(ignored): request.finish() d.addErrback(err, "Streaming data to client failed") d.addCallback(finished) return NOT_DONE_YETSorry for the earlier mis-information.
Jean-Paul
Twisted-web mailing list Twisted-web@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-web
Thanks again Jean-Paul,
This one will take me some time to digest and prototype. I'll give it a whirl to try to implement a solution and I'll post back my results.
dada! This one works!
import os from StringIO import StringIO from twisted.python.log import err from twisted.internet import reactor from twisted.web.server import Site, NOT_DONE_YET from twisted.web.resource import Resource from twisted.protocols.basic import FileSender
data = os.urandom(10*1024*1024)
class TestPage(Resource): isLeaf = True
def _cb_chunk_done(self, ignore, request): request.finish()
def render_GET(self, request): producer = FileSender() d = producer.beginFileTransfer(StringIO(data), request) d.addCallback(self._cb_chunk_done, request) d.addErrback(err, "Streaming data to client failed") return NOT_DONE_YET
root = Resource() root.putChild('test', TestPage()) reactor.listenTCP(8880, Site(root)) reactor.run()
Thanks again for your time, much appreciated!
On Dec 21, 2010, at 2:24 PM, exarkun@twistedmatrix.com wrote:
Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
While everybody should of course use producers and consumers, I feel like there should be a twisted core ticket for this behavior of transport buffering, and a twisted web ticket for this behavior of the request buffering. The naive implementation _could_ be much cheaper memory-wise; at the very least, twisted.web.static.Data ought to do the smart thing.
Would there be some deleterious effect to another usage pattern to fixing this? AFAIK there's no network in the world where you'll ever _actually_ be able to send more than about 64k in one send() anyway, and Twisted doesn't even try to do more than 128k (c.f. twisted.internet.abstract.FileDescriptor).
On 22 Dec, 06:51 am, glyph@twistedmatrix.com wrote:
On Dec 21, 2010, at 2:24 PM, exarkun@twistedmatrix.com wrote:
Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
While everybody should of course use producers and consumers, I feel like there should be a twisted core ticket for this behavior of transport buffering, and a twisted web ticket for this behavior of the request buffering. The naive implementation _could_ be much cheaper memory-wise; at the very least, twisted.web.static.Data ought to do the smart thing.
Fixing Data sounds like a good idea. I don't know what improvement to the transport buffering you're thinking of, though. It doesn't seem like there is an obvious, generally correct fix.
Jean-Paul
Cross-posting to Twisted core list because this is starting to get into general non-web stuff. Future replies about this fork of the thread should go there.
On Dec 23, 2010, at 11:22 AM, exarkun@twistedmatrix.com wrote:
On 22 Dec, 06:51 am, glyph@twistedmatrix.com wrote:
On Dec 21, 2010, at 2:24 PM, exarkun@twistedmatrix.com wrote:
Instead, you have to go all the way to producers/consumers, and only write more data to the transport buffer when it has finished dealing with what you previously gave it.
While everybody should of course use producers and consumers, I feel like there should be a twisted core ticket for this behavior of transport buffering, and a twisted web ticket for this behavior of the request buffering. The naive implementation _could_ be much cheaper memory-wise; at the very least, twisted.web.static.Data ought to do the smart thing.
Fixing Data sounds like a good idea. I don't know what improvement to the transport buffering you're thinking of, though. It doesn't seem like there is an obvious, generally correct fix.
Right now, FileDescriptor.write appends its input data directly to _tempDataBuffer. So far, so good: no string mangling.
So let's say we do fd.write(header); fd.write(veryBigBody).
Then we start writing.
FileDescriptor.doWrite comes along and notices that the dataBuffer is empty. The first thing it does in this case: ''.join(_tempDataBuffer); which copies the entire veryBigBody.
FileDescriptor is kinda sorta trying to avoid this problem by maintaining an 'offset' so it doesn't need to re-copy dataBuffer; it could use a similar tactic and do write()s out of individual chunks which are greater than SEND_LIMIT directly, rather than coalescing them together.
Or maybe we could wrap writev() instead of copying stuff at all?
On Dec 24, 2010, at 1:13 AM, Glyph Lefkowitz wrote:
Or maybe we could wrap writev() instead of copying stuff at all?
What an awesome idea!
http://twistedmatrix.com/trac/changeset/15847/branches/--omg-optimized-2#fil...
James
On Dec 24, 2010, at 11:47 AM, James Y Knight wrote:
On Dec 24, 2010, at 1:13 AM, Glyph Lefkowitz wrote:
Or maybe we could wrap writev() instead of copying stuff at all?
What an awesome idea!
http://twistedmatrix.com/trac/changeset/15847/branches/--omg-optimized-2#fil...
Doesn't look like that branch has gotten some love in quite a while :).
Are each of the features there (crecv, iovec, cdefer) represented by a different ticket already?
-
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
James Y Knight -
Pedro I. Sanchez