Gabriel Rossetti wrote:
Hello,
I have been trying to use the producer/consumer paradigm to transfer large strings. I've been having trouble because I have strange things going on, but I now suspect it isn't the P/C that's not working, because it works fine with smaller strings than the ones I'm having trouble with, so I suspect the problem comes from elsewhere. What happens is it starts off fine, but then the destination at some point only gets part of the string but the source has already sent everything, closed it's transport and called stopProducing(). Could there be some internal buffering problem, like I'm sending too much data at too high of a throughput, or maybe something like some sort of max time a socket can stay open or something? If I tell it to split the string up into smaller parts, it doesn't change anything, if I make the string smaller, whatever the size of each chunk, it transfers fine.
Thanks, Gabriel
I removed the P/C code and used the previous version and the problem persists, if I send a XML msg (very small XML part) with a data payload containing a string generated like this : initialMsg = "".join([str(x) for x in range(5000)]) which is 10374 bytes of data (including the XML), this makes me suspect that there is maybe some sort of buffer or something that is full and doesn't have the time to empty and twisted stops working or something. I'm using the 2.5 version since Ubuntu hasn't upgraded (and apparently won't upgrade until the lenny) their version yet and I can't get it to compile from source (yes, I installed python-dev and build-essentials), so maybe it's a problem only to this version. Does anyone have an Idea of what the problem is? I'm using the following code to send messages, I don't think that's the problem but you never know : def sendMessage(address, port, message, needAnswer=False): d = defer.Deferred() if needAnswer else None class MessageSender(Protocol): def sendMessage(self, msg): if domish.IElement.providedBy(msg): msg = msg.toXml() if isinstance(msg, unicode): msg = msg.encode('utf-8') self.transport.write(msg) def dataReceived(self, data): d.callback(data) def gotProtocol(proto): proto.sendMessage(message) if(not needAnswer): proto.transport.loseConnection() c = ClientCreator(reactor, MessageSender) c.connectTCP(address, port).addCallback(gotProtocol) return d Gabriel