[ python-Bugs-856706 ] popen3 under threads reports different stderr results

[SourceForge.net]

Bugs item #856706, was opened at 2003-12-09 04:08
Message generated for change (Comment added) made by facundobatista
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=856706&group_id=5470

Category: Threads
Group: Python 2.2.2
>Status: Closed
>Resolution: Duplicate
Priority: 5
Submitted By: Steven Howe (stevenhowe)
Assigned to: Nobody/Anonymous (nobody)
Summary: popen3 under threads reports different stderr results

Initial Comment:
Using os.popen3 inside a thread (start_new_thread)
returns different results from the stderr file dev.

I discovered this while trying to trap badblocks
program output. I have usb floppy I am trying to
format, after a badblock check. My floppy is on /dev/sdb.

code:
import thread
import os
def dd( outfilename ):
   cmd='/sbin/badblocks -n -s -v -c 16 -b 512 /dev/sdb'
   channels = os.popen3( cmd )
   ch = ' '
   ff = '/tmp/%s' % outfilename
   out=open( ff ,'w')
   while ch != '':
      ch = channels[2].read(1)
      out.write( ch )

Run two ways. First as a stand alone code. Then as a
threaded program.  

def( 'nothread.err' )
thead.start_new_thread( def, ( 'thread.err' ) )
---------------
Now view the results with od -ta. You will see that the
results are very different. All the the verbose data,
on current blocks completed, are missing.

Steven Howe


      

----------------------------------------------------------------------

>Comment By: Facundo Batista (facundobatista)
Date: 2005-01-15 16:59

Message:
Logged In: YES 
user_id=752496

Duplicate of #853411 (the OP says so in the other bug).

----------------------------------------------------------------------

Comment By: Andrew Gaul (gaul)
Date: 2003-12-25 16:42

Message:
Logged In: YES 
user_id=139865

This appears to be a duplicate of #853411.  The thread on
python-dev with subject "Python threads end up blocking
signals in subprocesses" discusses this.

----------------------------------------------------------------------

Comment By: Steven Howe (stevenhowe)
Date: 2003-12-12 16:02

Message:
Logged In: YES 
user_id=916892

Hello Gaul. Well I found a work around. Using
threading.Thread, I init a routine that forks and execv's a
script (execv has no method to accept redirects '> 2>') that
runs the badblocks program and route output to files. 

Then I the a thread that uses open() to attach the <stderr>
to a progress reading routine and when complete <stdout> for
the badblock list (if any).

This method created another problem. Popen3 does not return
an end of file ( '' ) until the process has ended. With
badblocks forked, there is no syncronization between my
script and the badblocks output. So I can and do overrun the
<stderr> output, which then returns an EOF.  
Another workaround:
I wrote a routine to make sure I never read to the end of
file:     
     readsize = os.stat(fname)[stat.ST_SIZE] - fptr.tell() 
- BIAS

All this so I can using threading. No doubt you're asking
why use threading? I'm making a pygtk app similar to
'gfloppy' that can handle USB floppies. I need to make a
progress meter. Using threading allows a GTK call back to
examine the current status of badblocks.
But a fix would be preferable.  Thanks,

Steven Howe

----------------------------------------------------------------------

Comment By: Andrew Gaul (gaul)
Date: 2003-12-12 07:03

Message:
Logged In: YES 
user_id=139865

Confirmed with Python CVS and Fedora on x86. 
e2fsprogs/misc/badblocks:alarm_intr sets a SIGALRM handler
and calls alarm(1), but it is not getting fired.  I will
look into this further.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=856706&group_id=5470