Interprocess communication woes

[Murali]

On Jul 19, 4:30 am, Nick Craig-Wood <n... at craig-wood.com> wrote:
> Murali <murali.dhanak... at gmail.com> wrote:
> >  After some investigation, I found out that this problem had nothing to
> >  do with my GUI app not getting refreshed and I was able to reproduce
> >  this problem with normal python scripts. Here is one such script
>
> >  #File test.py
> >  from subprocess import Popen
> >  from subprocess import PIPE
> >  import sys
> >  if __name__ == '__main__':
> >   prog = sys.argv[1]
> >   proc = Popen(prog, shell = True, stdout = PIPE, bufsize = 0)
> >   out = proc.stdout
> >   while proc.poll() is None:
> >     print out.readline()
>
> >  Run this program as follows
> >  $ test.py "ping -c 10www.google.com"
>
> >  Now, you would see the responses as if you just launched ping on the
> >  command line. Now, lets look at another program. Here is a simple C++
> >  program that prints numbers 1 to 10 at the passage of every second
> >  (sort of a stopwatch)
>
> >  #include <stdio.h>
> >  #include <stdlib.h>
> >  #include <sys/time.h>
> >  main ( )
> >  {
> >   timeval t1, t2;
> >   gettimeofday(&t1, NULL);
> >   int prev = -1;
> >   int cur = 0;
> >   while (true)
> >   {
> >     gettimeofday(&t2,NULL);
> >     if(t2.tv_sec - t1.tv_sec > 10)
> >       break;
> >     else
> >     {
> >       cur = t2.tv_sec-t1.tv_sec;
> >       if (cur != prev)
> >       {
> >         printf("%d\r\n",cur);
> >         prev = cur;
> >       }
> >     }
> >   }
> >  }
>
> >  if you would build this program and call it lets say timer ($ g++ -o
> >  timer timer.cpp)  and run it with our python script like this
>
> >  $python test.py "./timer"
>
> >  you would see that every time you run the program your results vary
> >  and on top of this the stdout of the timer program gets displayed all
> >  at once presumably when the timer program has completed execution.
>
> >  Why this discrepancy between the ping and timer programs? Is my
> >  test.py script correct? Is there a better or a preferred method for
> >  doing interprocess communication in Python.
>
> Buffering is your problem.
>
> If you add a fflush(stdout); after the printf(...); you'll find the
> c++ program works as you expect.
>
> It is just a fact of life of the C stdio system.  If it is connected
> to a terminal then it will turn off buffering.  If it is connected
> anything else (eg a pipe via subprocess) then it will buffer stuff as
> you've seen.
>
> So you can
>
Thanks Nick. fflush fixed it. Thanks for your pointers on pexpect and
pty module too.

Murali.


> a) modify the c++ prog to add fflush() in or use setvbuf()
> b) use the pexpect module -http://pexpect.sourceforge.net/
> c) use the pty module (unix only)
>
> The pexpect module will connect to the subprogram with pseudo-ttys,
> fooling the program, and the C library, into thinking that it is
> speaking to a terminal and turn off buffering.  Pexpect doesn't work
> on windows.
>
> The fact that ping works is because it uses fflush() - you can see
> this if you "ltrace" it.
>
> --
> Nick Craig-Wood <n... at craig-wood.com> --http://www.craig-wood.com/nick