[Twisted-Python] Question about SerialPort

Hi! I am trying to read Modbus RTU messages over the serial line. There is a library called pyModbus which is very usable for Modbus TCP/IP, but the support for RTU is a bit lacking. The main problem seems to be the specification of Modbus RTU. RTU messages are separated by time gaps that are at least 3.5 characters long. Unfortunately, this is the only reliable way for separating messages - there is no length header or something similar. The specification does define a trailing CRC, though. As far as I understand the event loop, I will not be able to guarantee that will not miss such a gap - the reactor might be busy handling some other events while the gap arrives. But when the reactor assigns some time to read from the serial port, I'd like it to read until the gap is detected. My current attempt is to override the doRead method of SerialPort by something like this: def doRead(self): port = self.fileno() result = [] while True: # The following code is stolen from fdesc.readFromFD try: output = os.read(port, 1) if output: result.append(output) else: break except (OSError, IOError), ioe: if ioe.args[0] in (errno.EAGAIN, errno.EINTR): return else: return CONNECTION_LOST if not result: return CONNECTION_DONE else: self.protocol.rawDataReceived("".join(result)) If the serial port is configured with a timeout about the size of the gap, os.read() should return an empty string, thus indicating that the message is complete. Most of the code above is a copy of t.i.fdesc.readFromFD. The main difference is that I don't read 8192 bytes, but only one at a time, to detect a time gap. Does this make sense? Do you think it would be possible to replace the constant 8192 in readFromFD by a parameter (that defaults to 8192)? This would allow me to replace the content of the while loop with a simple function call. TIA and best regards, Albert -- Albert Brandl Weiermayer Solutions GmbH | Abteistraße 12, A-4813 Altmünster phone: +43 (0) 720 70 30 14 | fax: +43 (0) 7612 20 3 56 web: http://www.weiermayer.com

On Jan 20, 2011, at 6:32 AM, Albert Brandl wrote:
The main problem seems to be the specification of Modbus RTU. RTU messages are separated by time gaps that are at least 3.5 characters long. Unfortunately, this is the only reliable way for separating messages - there is no length header or something similar. The specification does define a trailing CRC, though.
I don't think you'll ever be able to do this reliably in Python. At 9600 baud, 3.5 characters is just about 275hz (9600/10/3.5). Linux schedules process at around 500hz. So, unless you use real-time scheduling, I think you have no guarantee that you'll be able to detect such a gap at all -- some other process may be running and you may completely miss it. And you can't reasonably use real-time scheduling in python... On the other hand, are you sure you need this? http://www.libmodbus.org/ seems to have code which figures out what length packets should be, and no code to detect a 3.5char delay between packets. That seems much more reliable. It also doesn't seem to have any code to ensure a 3.5char between sent packets, which indicates to me that either it's totally brokem or else that not even the hardware end actually uses a 3.5char delay to determine packet boundaries...I'd guess the second, since even on an embedded CPU, it's much trickier to ensure the appropriate serial port timing than to simply count the correct number of bytes. James

On Thu, Jan 20, 2011 at 08:42:58AM -0500, James Y Knight wrote:
I don't think you'll ever be able to do this reliably in Python. At 9600 baud, 3.5 characters is just about 275hz (9600/10/3.5). Linux schedules process at around 500hz.
Well, that's bad news - I'll have to communicate with the SPS at 38400 baud :-(...
On the other hand, are you sure you need this? http://www.libmodbus.org/ seems to have code which figures out what length packets should be, and no code to detect a 3.5char delay between packets. That seems much more reliable.
Thanks for the hint - I'll have a look. The code seems quite readable, although I don't exactly _love_ reading C sources. But the idea that one might still find out where a modbus message ends is intriguing - I don't yet see how this might work.
It also doesn't seem to have any code to ensure a 3.5char between sent packets
Not sure about this - there is a structure called ctx which has a timeout_begin->tv_sec and timeout_begin->tv_usec (and a similar thing for timeout_end). On the first glance, this looks like variables for handling the boundary. But I might be wrong. Thanks & best regards, Albert -- Albert Brandl Weiermayer Solutions GmbH | Abteistraße 12, A-4813 Altmünster phone: +43 (0) 720 70 30 14 | fax: +43 (0) 7612 20 3 56 web: http://www.weiermayer.com

On 1/20/2011 9:42 PM, James Y Knight wrote:
On Jan 20, 2011, at 6:32 AM, Albert Brandl wrote:
The main problem seems to be the specification of Modbus RTU. RTU messages are separated by time gaps that are at least 3.5 characters long. Unfortunately, this is the only reliable way for separating messages - there is no length header or something similar. The specification does define a trailing CRC, though.
I don't think you'll ever be able to do this reliably in Python. At 9600 baud, 3.5 characters is just about 275hz (9600/10/3.5). Linux schedules process at around 500hz. So, unless you use real-time scheduling, I think you have no guarantee that you'll be able to detect such a gap at all -- some other process may be running and you may completely miss it. And you can't reasonably use real-time scheduling in python...
On the other hand, are you sure you need this? http://www.libmodbus.org/ seems to have code which figures out what length packets should be, and no code to detect a 3.5char delay between packets. That seems much more reliable.
It also doesn't seem to have any code to ensure a 3.5char between sent packets, which indicates to me that either it's totally brokem or else that not even the hardware end actually uses a 3.5char delay to determine packet boundaries...I'd guess the second, since even on an embedded CPU, it's much trickier to ensure the appropriate serial port timing than to simply count the correct number of bytes.
Building upon James' reply a little further, I recently did a ModBus RTU implementation on a microcontroller and you need both a 1.5 and 3.5 character timer for correct operation. With the micro, I used hardware timers with uS resolution synchronized to incoming characters via the UART interrupt. As James mentions, user-space Python is simply too far from the hardware to do this sort of thing. You'd probably have to write a custom UART driver and run it on a RTOS to get correct timing. That being said, you can probably write something that will work 99.9% of the time, but may not respond correctly for some weird corner case scenarios. Jason Valenzuela
participants (3)
-
Albert Brandl
-
James Y Knight
-
Jason Valenzuela