[Twisted-Python] Slow data transfer with Twisted + socat + Windows
I'm writing an application with Twisted/GTK to manage a serial device, for use on both Windows and Linux. Unfortunately I can't use serial comms directly under Windows[1], so my workaround is to launch socat[2] in a subprocess to relay data between a local TCP port and the serial port. The socat command I use is: socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600 I can then send and receive data over localhost:31415 same as I would for a serial port. The trouble is, a transaction that takes a minute under Linux (using *either* direct serial comms or the same socat subprocess approach) now takes more than 50 times as long under Windows. I've attached the script I'm using to test, mainly so it's clear what I'm doing — although it's useless unless you happen to have a serial device using a protocol you can shoehorn into that script. I've also attached the output where you can see the difference between different reactors on different platforms. (In my app I use reactor.spawnProcess(...), but I'm not doing that here.) Basically, under Linux either approach takes 4 seconds; under Windows the SelectReactor takes 30s, and the gtk2reactor takes 220s! Profiling the "reactor.run()" call on the different platforms (both for the select reactor) showed that (a) most time was spent in the select.select call, and (b) the time spent in select was 8 times more under Windows than Linux. I have no idea if this is useful information or not. I'm pretty sure it's not a socat-specific issue, because I get the same problem if I use com2tcp[3]. It could be Cygwin related, but I don't know how determine that for sure. Also, the fact that it runs slower using the gtk2reactor (vs. selectreactor) suggests to me that it could be my code, not theirs. I found an old Cygwin thread[4] that claimed setting NODELAY on the listening socket helped with a similar problem, but using > socat TCP4-LISTEN:31415,nodelay [etc] ...made no difference. So I'm a bit stuck. Does anyone know how I can narrow it down further? Has someone else noticed slow TCP behaviour under Windows? Or Cygwin utilities? Have I made some glaringly obvious mistake in my script there? Any help appreciated. :) Cheers, Jason [1] http://twistedmatrix.com/trac/ticket/4862 [2] http://www.dest-unreach.org/socat/ [3] http://com0com.sourceforge.net/ [4] http://www.mail-archive.com/cygwin@cygwin.com/msg66791.html
On 8 March 2011 08:38, Jason Heeris <jason.heeris@gmail.com> wrote:
I'm writing an application with Twisted/GTK to manage a serial device, for use on both Windows and Linux. Unfortunately I can't use serial comms directly under Windows[1], so my workaround is to launch socat[2] in a subprocess to relay data between a local TCP port and the serial port. The socat command I use is:
socat TCP4-LISTEN:31415 /dev/com4,raw,echo=0,b57600
I can then send and receive data over localhost:31415 same as I would for a serial port.
The trouble is, a transaction that takes a minute under Linux (using *either* direct serial comms or the same socat subprocess approach) now takes more than 50 times as long under Windows.
I've attached the script I'm using to test, mainly so it's clear what I'm doing — although it's useless unless you happen to have a serial device using a protocol you can shoehorn into that script. I've also attached the output where you can see the difference between different reactors on different platforms. (In my app I use reactor.spawnProcess(...), but I'm not doing that here.)
Basically, under Linux either approach takes 4 seconds; under Windows the SelectReactor takes 30s, and the gtk2reactor takes 220s!
Profiling the "reactor.run()" call on the different platforms (both for the select reactor) showed that (a) most time was spent in the select.select call, and (b) the time spent in select was 8 times more under Windows than Linux. I have no idea if this is useful information or not.
I'm pretty sure it's not a socat-specific issue, because I get the same problem if I use com2tcp[3]. It could be Cygwin related, but I don't know how determine that for sure. Also, the fact that it runs slower using the gtk2reactor (vs. selectreactor) suggests to me that it could be my code, not theirs.
I found an old Cygwin thread[4] that claimed setting NODELAY on the listening socket helped with a similar problem, but using
> socat TCP4-LISTEN:31415,nodelay [etc]
...made no difference.
So I'm a bit stuck. Does anyone know how I can narrow it down further? Has someone else noticed slow TCP behaviour under Windows? Or Cygwin utilities? Have I made some glaringly obvious mistake in my script there?
Does you program just write via TCP to this serial device, it doesn't do anything else that could block? Have you tried running this under different reactors? Running the windows program against the Linux socat device might help to reduce the number of variables. Michael
On 9 March 2011 03:49, Michael Thompson <michaelnt@gmail.com> wrote:
Does you program just write via TCP to this serial device, it doesn't do anything else that could block?
Both the "bare bones" script and the real app writes and reads from the device, so I wouldn't expect full speed communications. But I can't see anything else in my script that might actually block.
Have you tried running this under different reactors?
Yep — no difference under Linux, but the GTK reactor is slower than the select reactor under Windows (see original email for times).
Running the windows program against the Linux socat device might help to reduce the number of variables.
Aha! Didn't think of that, but interesting results... socat on Linux, socat_test.py on Windows VM: - select reactor: 4s - win32reactor: 4s - gtk reactor: 220s socat on Windows VM, socat_test.py on Linux: - select reactor: 30s - gtk reactor: 30s Interesting, but not exactly clarifying... — Jason
participants (2)
-
Jason Heeris -
Michael Thompson