Mailman 3 March 2005 - Twisted

[Twisted-Python] streaming producer
by Andrea Arcangeli 09 Mar '05

09 Mar '05

Could somebody explain the semantics of the streaming parameter to registerProducer? I believe I have to set it to 1. I need to stop data coming from a process protocol, and to do so I need to register a producer somehow. I need to stop the process protocol when the TCP transport buffer (or the sendstring buffer or both? I'm using int32protocol) grows too much (let's say 10mbytes, I like to allow huge buffering, but at the same time I can't avoid the producer or people could trigger oom conditions trivially). I don't understand exactly how I have to setup the thing. I've two protocols, one is the tcp protocol, the other is the process protocol. I will probably make the max buffer size configurable dynamically and accounted in the ram_mbytes selection as well. It seems twcgi has the very same issue, so I'm taking ideas from there ;). Thanks!

1 1

RE: [Twisted-Python] TCP Packet Size Considerations: Extra Info
by Grant McDonald 09 Mar '05

09 Mar '05

Now i do feel stupid! The documentation doesn't mention anything about sending multiple packets..... how can i tell if a packet is part of a multiple packet set? After looking at the output more carefully i realised that it was sending a packet the size of 8760 and 8077 which together total 16837. Where is the information to say it is a split transmission? -----Original Message----- From: twisted-python-bounces(a)twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com]On Behalf Of Grant McDonald Sent: 09 March 2005 16:56 To: 'twisted-python(a)twistedmatrix.com' Subject: [Twisted-Python] TCP Packet Size Considerations: Extra Info After adding some code to report the transmission size i can confirm that 16837 bytes are sent and 8077 bytes are received via the previous code. Regards, Grant Important notice: This message is intended for the individual(s) and entity(s) addressed. The information contained in this transmission and any attached, may be confidential and may also be the subject of legal privilege, public interest immunity or legal professional privilege. Any review, retransmission, dissemination or other use of, taking of any action in reliance upon this information by person or entities other than the recipient is prohibited and requires authorization from the sender. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person) you may not copy or deliver this message to anyone. In such cases you should destroy this message and kindly notify the sender by reply email. WARNING: Although Infocomp has taken reasonable precautions so that no viruses are present in this e-mail, the company cannot accept responsibility for any loss or damage arising from the use of e-mail attachments.

2 1

[Twisted-Python] TCP Packet Size Considerations: Extra Info
by Grant McDonald 09 Mar '05

09 Mar '05

After adding some code to report the transmission size i can confirm that 16837 bytes are sent and 8077 bytes are received via the previous code. Regards, Grant

2 1

RE: [Twisted-Python] TCP Packet Size Considerations
by Grant McDonald 09 Mar '05

09 Mar '05

JP wrote: > There are no such known limits. I might even go as far to say that there > are no such limits, if I were feeling daring. :) Do you have a short > program which demonstrates this behavior that you can share? > Perhaps after looking at that someone can suggest the cause of the disconnects. After looking into it some more myself i realised an uncaught exception was dropping the connection but the after cutting my test code back to minimal functionality the inconsistency with received data was still there. The last 4KB of the data was received and the first 14KB discarded somewhere! I have created a minimal example from my initial test code. Start the simpleserver with twistd.py -y simpleserver.py and just run the simpleclient.py the builder.dat file is just a nonsense file that is about 18KB in size. Testing on my machine produced a received.dat file which was 4KB in size. I am currently using Python 2.3.3 and Twisted 1.3.0. The code is as follows (and is also attached for convenient running): ### ------------ simpleclient.py --------------------------------- from twisted.internet.protocol import Protocol import twisted.internet.error from simpleprotos import SimpleClientFactory factory = SimpleClientFactory() # open a large nonsense file to send (18KB) f = file('builder.dat') data = f.read() print len(data) f.close() from twisted.internet import reactor reactor.connectTCP('127.0.0.1', 10000, factory) reactor.callLater(1.0, factory.sendMessage, data) reactor.run() ### ------------- end --------------------------------------------- ### ------------ simpleserver.py ---------------------------------- from twisted.application import internet # services that run TCP/SSL/etc. from twisted.application import service from simpleprotos import ServerDataConnection, SimpleFactory def makeService(): factory = SimpleFactory() # here we create a ConnectionManager return internet.TCPServer(10000, factory) application = service.Application("Connection Manager") sc = service.IServiceCollection(application) sc.addService(makeService()) ### ------------- end --------------------------------------------- ### ------------ simpleprotos.py ---------------------------------- from twisted.internet.protocol import ClientFactory, Factory, Protocol class SimpleClientFactory(ClientFactory): def buildProtocol(self, addr): self.p = ClientDataConnection() self.p.factory = self return self.p def sendMessage(self, data): if hasattr(self, 'p'): self.p.sendData(data) class SimpleFactory(Factory): def buildProtocol(self, addr): self.p = ServerDataConnection() self.p.factory = self return self.p class ServerDataConnection(Protocol): def closeConnection(self): self.transport.loseConnection() def dataReceived(self, data): f = file('received.dat', 'w') f.write(data) f.close() class ClientDataConnection(Protocol): def closeConnection(self): self.transport.loseConnection() def sendData(self, data): self.transport.write(data) ### ------------- end --------------------------------------------- Regards, Grant McDonald

1 0

[Twisted-Python] TCP Packet Size Considerations
by Grant McDonald 09 Mar '05

09 Mar '05

Hi All, I am translating a twisted UDP implementation to a TCP implementation and i have come across some unexpected errors related to the amount of data written to a TCP transport. When a large amount of data is written to a TCP connection (approx. 9KB) the connection is dropped and transmission is unsuccessful. Between about 4KB-9KB (approx.) the connection is not dropped but the transmission is incomplete as the unpickling raises a ValueError (insecure pickle string). Under 4KB transmission is faultless. Is this a known limitation? I was aware of the 8KB of UDP packets but assumed that there were no such limitations on TCP connections. Can someone please enlighten me? Regards, Grant McDonald PS. these are very approximate sizes - if need be i can narrow them down some more.

2 1

[Twisted-Python] Re: [Twisted-commits] r13090 - Use ThreadedResolver instead of BlockingResolver by default - the thread import issue is moot, PyOpenSSL imports thread unconditionally.
by Itamar Shtull-Trauring 09 Mar '05

09 Mar '05

On Mon, 2005-03-07 at 22:31 -0700, Jp Calderone wrote: > Use ThreadedResolver instead of BlockingResolver by default - the thread import issue is moot, PyOpenSSL imports thread unconditionally. Not true. Some BSDs don't do threading in Python, and we need to support them. Remember, the pyOpenSSL support is optional. The fact pyOpenSSL does the import unconditionally is a bug.

4 5

[Twisted-Python] Twisted Allen-Bradley PLC Protocol
by Paul Campbell 09 Mar '05

09 Mar '05

I've successfully gotten twisted to talk to Allen Bradley PLC 5's using Ron Gage's "ABEL" library as a template. Needless to say, the twisted way is cleaner and easier to maintain (in my opinion at least). However, there's a slight twist to the code and I'm trying to think if there's a better way, and the protocol is similar to so many other things (SSL comes to mind). To communicate with AB PLC's, first you initiate a TCP/IP connection. This is text book twisted. Second, you initiate communication using a "link layer" protocol. Once that is established, THEN communication progresses using yet another protocol layer ("PCCC" or "DF-1" or "CSP" depending on what document you are reading). Right now, I simply wrote my "connectionMade" function set self.attached=0 and then initiate a "send connect command" function. The "received connection" function sets self.attached=1. All other commands simply to a reactor.callLater(1,self.func) and float around in the reactor buffer until self.attach==1 returns with good news. It seems to me that what we have here is really a protocol stack, similar in nature to the "LineReceiver" type protocol or perhaps it should be set up as an adapter (component architecture). Then in this case, the packet/protocol gets passed up through the layers and the "link layer" occupies the correct status of being a lower level layer. Or else to support say serial communication on an RS-232 or RS-485 network later, it could be done as a component arrangement. So, does this make any sense? What is the "design pattern" in this case in twisted?

2 1

[Twisted-Python] wxreactor and dialog on win xp problem
by Michele Petrazzo 08 Mar '05

08 Mar '05

I have a problem with wxreactor on win xp. I insert a ShowModal() of a TextEntryDialog after an event of a button in a frame. The ShowModal function of the dialog, don't return me the control after the dialog has close, but only when the global application has closed. I try this with a bit of print. Show the example for tests. On my linux-box, with the same wxpython and twisted version, all work well. python 2.3.4 wx 2.5.3.1 twisted 1.3.0 Thanks, Michele

2 1

[Twisted-Python] Help With Twisted as a Client (newbie warning)
by Steve Milner 08 Mar '05

08 Mar '05

I'm trying to get a better handle on how to write a client using twisted that would react to a GUI. What I am wanting to do is to somehow be able to work with xmlclient (which is based in sux.XMLParser) so I can write to the transport. Here is the connection code: <snip> # Overrides the protocol.XlientFactory with a bit of reconnect code and sets # the protocol to xmlclient factory = CUClientFactory() factory.clients = [] sslFactory = ClientContextFactory() application = service.Application("xmlclient") reactor.connectSSL("__example__.com", 1026, factory, sslFactory) </snip> Is there a way I can access xmlxlient inside application? Is there a different way I should send data to the server? I can send data if I explicitly say so inside xmlclient, but what I am wanting to do is to send it due to outside sources (like a button click or key press). Something like this: <bad code> application.xmlparser.transport.write("testing") </bad code> I've spent a few hours searching through google and the examples/docs but haven't found out how to do this (or even if it is the right line of thinking). All help is greatly appreciated! Thanks, Steve

2 3

RE: [Twisted-Python] watch a directory (Win32)
by Henning.Ramm＠mediapro-gmbh.de 08 Mar '05

08 Mar '05

>... and call it with reactor.callLater(0,someFunctionThatCallsWaiting) >regularily Oh, sure, thanks. I just didn't understand how stupid and non-magic these Win32 functions are. Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

Twisted March 2005