[Twisted-Python] Guidance needed on serial device interaction
This is a bit long, sorry... I have a PyGTK program that uses threads and pyserial's blocking methods to interact with an RS232 connected device. I'd like to throw out the threading awfulness and redo it in Twisted, if possible, but I'm a little lost. The real protocol is a bit convoluted, but basically: - You can issue single character commands to the device that give a fixed length response, such as sending 'C' and getting an eight-digit hex string back (the program flash CRC) - You can put the device into "programming mode" (command 'P'), where it takes an arbitrary length sequence of records, verifying each record and stopping when it sees the special "end record" - The device will send back '!' to indicate an error - The device will send back '>' to indicate that it's ready for more commands I know I need to use the SerialPort transport, and since that takes a protocol I've tried to sketch one out but can't seem to get very far. I want to have methods that can be called from the UI, returning deferreds to which UI responses can be added as callbacks. I also want to be able to monitor progress of the programming, so I need a callback for that separate from the deferred itself. I figure that calling, say, device.program(program, progress_cb) should return a Deferred and queue the command, and that somehow the dataReceived() method should be the start of a chain of events that either: - updates state and waits for the next piece of data, or - calls back on the appropriate Deferred I'd also like to structure things so that successive calls to the DeviceProtocol object queue up, something like: ---- class DeviceProtocol(Protocol): def dataReceived(self, data): # ...? def checksum(self): res = defer.Deferred() # When ready, send 'C' over the serial line (ie. # self.transport.write('C')) # Somehow callback via a deferred when the checksum comes back return res def program(self, program_records, progress_cb): res = defer.Deferred() # When ready, send 'P' over the serial line # Write all the data in "program_records", checking each # response, calling back via a deferred when done. # ... return res def connectionLost(self): # maybe have something here, like calling the errback of all # pending deferreds def go(reactor): # callbacks/errbacks not shown device = DeviceProtocol() transport = SerialPort(device, portname, reactor) program_result_1 = device.program(records, good_programming_progress) program_result_1.addCallbacks(programming_done, programming_err) checksum_result = device.checksum() checksum_result.addCallbacks(checksum, checksum_err) # This will fail at some point: program_result_2 = device.program(bad_records) cp_deferred.addCallbacks(programming_done, programming_err) if __name__ == "__main__": reactor.callWhenRunning(go, reactor) reactor.run() ---- But I'm really lost as to how to start structuring things within the protocol object itself. Should I even be doing this in a Protocol subclass, or should I be putting some of this functionality into a Factory of some sort (eg. a ClientFactory)? And if so, how do I actually connect the factory to the serial port transport? Or am I on the wrong track altogether? More to the point, has a problem like this already been solved somewhere? Is the solution obvious to Twisted gurus? Any help would be appreciated. Cheers, Jason
On 2 February 2011 06:36, Jason Heeris <jason.heeris@gmail.com> wrote:
This is a bit long, sorry...
I have a PyGTK program that uses threads and pyserial's blocking methods to interact with an RS232 connected device. I'd like to throw out the threading awfulness and redo it in Twisted, if possible, but I'm a little lost.
The real protocol is a bit convoluted, but basically: - You can issue single character commands to the device that give a fixed length response, such as sending 'C' and getting an eight-digit hex string back (the program flash CRC) - You can put the device into "programming mode" (command 'P'), where it takes an arbitrary length sequence of records, verifying each record and stopping when it sees the special "end record" - The device will send back '!' to indicate an error - The device will send back '>' to indicate that it's ready for more commands
The job of the protocol class is to assemble the bytes that you receive into packets or messages. Ideally the protocol will have characters that frame a message, and it sounds like you might have this if > or ! is always sent at the end of every reply. The protocol dataReceived method will be called as data arrives at the serial port once enough data has arrived to complete a message this method calls stringReceived with the complete message. There are some protocol that might be suitable in twisted/protocols/basic.py
<snip> I'd also like to structure things so that successive calls to the DeviceProtocol object queue up, something like:
You can use a DeferredSemaphore for this. <snip> Michael
On 2 February 2011 15:58, Michael Thompson <michaelnt@gmail.com> wrote:
The job of the protocol class is to assemble the bytes that you receive into packets or messages. Ideally the protocol will have characters that frame a message, and it sounds like you might have this if > or ! is always sent at the end of every reply.
Apologies, but I just went over the code for the device controller and I described the protocol a little incorrectly. But anyway... It depends on what you mean by reply. For example a programming session might go: Dev: ">" PC: "P" Dev: "." PC: ":" Dev: "." PC: "02000004" Dev: "." PC: "1D00" Dev: "." PC: "DD" Dev: "." <skip to last transmission for last record> PC: "FF" Dev: ".0>" (this is the confirmation ".", the status code of the programming, and the "ready" character) It's not like the PC sends all the data at once, and the device processes it all and sends a reply framed by ">". If something goes wrong in the device, it might be: Dev: ">" PC: "P" Dev: "1>" ...or if the sent data is invalid... Dev: ">" PC: "P" Dev: "." PC: ":" Dev "." PC: "09" Dev: "3>"
The protocol dataReceived method will be called as data arrives at the serial port once enough data has arrived to complete a message this method calls stringReceived with the complete message.
I don't know what this is for though... what's the point of stringReceived? What uses that?
There are some protocol that might be suitable in twisted/protocols/basic.py
No, that all seems to be unrelated to what I need.
I'd also like to structure things so that successive calls to the DeviceProtocol object queue up, something like:
You can use a DeferredSemaphore for this.
That looks interesting, thanks :) — Jason
Hi! On Wed, Feb 02, 2011 at 04:58:30PM +0800, Jason Heeris wrote:
The protocol dataReceived method will be called as data arrives at the serial port once enough data has arrived to complete a message this method calls stringReceived with the complete message.
I don't know what this is for though... what's the point of stringReceived? What uses that?
"string" could be interpreted as "complete message". It might e.g. happen that a message arrives in three chunks. Each time a chunk is read, the dataReceived method is called. When it detects that the message is complete, it calls stringReceived with the content of the message. Beware that the third chunk might also contain a part of the next message. This should of course _not_ be forwarded to stringReceived, but buffered until the next message is complete. It helped me a lot to read the source code in basic.py, which can be found in the directory twisted/protocols, or online here: http://twistedmatrix.com/trac/browser/tags/releases/twisted-10.2.0/twisted/p... It has lots of documentation and is very readable IMHO. Regards, Albert -- Albert Brandl Weiermayer Solutions GmbH | Abteistraße 12, A-4813 Altmünster phone: +43 (0) 720 70 30 14 | fax: +43 (0) 7612 20 3 56 web: http://www.weiermayer.com
On 2 February 2011 17:53, Albert Brandl <albert.brandl@weiermayer.com> wrote:
"string" could be interpreted as "complete message". It might e.g. happen that a message arrives in three chunks. Each time a chunk is read, the dataReceived method is called. When it detects that the message is complete, it calls stringReceived with the content of the message.
Okay, but I don't see how to use that to solve my particular problem. I'm not waiting passively to receive a complete string, I have to react to whatever's sent back, character by character, either by reporting completion, an error or sending more data. In effect, I guess, each character is a "complete message" anyway. I don't think the t.i.protocols offer much for that. — Jason
On 2 February 2011 10:11, Jason Heeris <jason.heeris@gmail.com> wrote:
On 2 February 2011 17:53, Albert Brandl <albert.brandl@weiermayer.com> wrote:
"string" could be interpreted as "complete message". It might e.g. happen that a message arrives in three chunks. Each time a chunk is read, the dataReceived method is called. When it detects that the message is complete, it calls stringReceived with the content of the message.
Okay, but I don't see how to use that to solve my particular problem. I'm not waiting passively to receive a complete string, I have to react to whatever's sent back, character by character, either by reporting completion, an error or sending more data. In effect, I guess, each character is a "complete message" anyway. I don't think the t.i.protocols offer much for that.
Yep you have a pretty simple protocol there so even the basic examples are probably more than you need. Something like this might get you started. class MyProtocol(Protocol): def send(self, byte): self.transport.write(byte) self.response = Deferred() return self.response def dataReceived(self, byte): self.response.callback(byte)
On Feb 2, 2011, at 7:42 AM, Michael Thompson wrote:
On 2 February 2011 10:11, Jason Heeris <jason.heeris@gmail.com> wrote:
On 2 February 2011 17:53, Albert Brandl <albert.brandl@weiermayer.com> wrote:
"string" could be interpreted as "complete message". It might e.g. happen that a message arrives in three chunks. Each time a chunk is read, the dataReceived method is called. When it detects that the message is complete, it calls stringReceived with the content of the message.
Okay, but I don't see how to use that to solve my particular problem. I'm not waiting passively to receive a complete string, I have to react to whatever's sent back, character by character, either by reporting completion, an error or sending more data. In effect, I guess, each character is a "complete message" anyway. I don't think the t.i.protocols offer much for that.
Yep you have a pretty simple protocol there so even the basic examples are probably more than you need.
Something like this might get you started.
class MyProtocol(Protocol): def send(self, byte): self.transport.write(byte) self.response = Deferred() return self.response
def dataReceived(self, byte): self.response.callback(byte)
Not quite. You mean: class MyProtocol(Protocol): # ... def dataReceived(self, data): for octet in data: self.process(octet) If the device outputs multiple bytes it might show up to Twisted as strings of arbitrary length (this depends on timings of the serial port which are basically impossible to control), so if you want to process a byte at a time, you need to specifically iterate through the string that is passed, even if it's usually one byte long. (Some of the protocol examples showed multibyte sequences as messages. I don't understand the framing well enough to provide a better example though, sorry.)
participants (4)
-
Albert Brandl -
Glyph Lefkowitz -
Jason Heeris -
Michael Thompson