[Python-checkins] python/dist/src/Doc/lib libos.tex,1.53.4.8,1.53.4.9 libpopen2.tex,1.13.6.1,1.13.6.2

fdrake@users.sourceforge.net fdrake@users.sourceforge.net
Tue, 18 Jun 2002 13:32:11 -0700


Update of /cvsroot/python/python/dist/src/Doc/lib
In directory usw-pr-cvs1:/tmp/cvs-serv15627/lib

Modified Files:
      Tag: release21-maint
	libos.tex libpopen2.tex 
Log Message:
Add description of the deadlock problem with child processes and pipes, and
hints about how to work around it.
Closes SF bug #530637.


Index: libos.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v
retrieving revision 1.53.4.8
retrieving revision 1.53.4.9
diff -C2 -d -r1.53.4.8 -r1.53.4.9
*** libos.tex	18 Jun 2002 16:17:31 -0000	1.53.4.8
--- libos.tex	18 Jun 2002 20:32:09 -0000	1.53.4.9
***************
*** 315,318 ****
--- 315,323 ----
  for \var{mode} is \code{'t'}.
  
+ For a discussion of possible dead lock conditions related to the use
+ of these functions, see ``\ulink{Flow Control
+ Issues}{popen2-flow-control.html}''
+ (section~\ref{popen2-flow-control}).
+ 
  \begin{funcdesc}{popen2}{cmd\optional{, mode\optional{, bufsize}}}
  Executes \var{cmd} as a sub-process.  Returns the file objects

Index: libpopen2.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/lib/libpopen2.tex,v
retrieving revision 1.13.6.1
retrieving revision 1.13.6.2
diff -C2 -d -r1.13.6.1 -r1.13.6.2
*** libpopen2.tex	6 Jul 2001 17:18:05 -0000	1.13.6.1
--- libpopen2.tex	18 Jun 2002 20:32:09 -0000	1.13.6.2
***************
*** 108,109 ****
--- 108,171 ----
  The process ID of the child process.
  \end{memberdesc}
+ 
+ 
+ \subsection{Flow Control Issues \label{popen2-flow-control}}
+ 
+ Any time you are working with any form of inter-process communication,
+ control flow needs to be carefully thought out.  This remains the case
+ with the file objects provided by this module (or the \refmodule{os}
+ module equivalents).
+ 
+ % Example explanation and suggested work-arounds substantially stolen
+ % from Martin von Löwis:
+ % http://mail.python.org/pipermail/python-dev/2000-September/009460.html
+ 
+ When reading output from a child process that writes a lot of data to
+ standard error while the parent is reading from the child's standard
+ out, a dead lock can occur.  A similar situation can occur with other
+ combinations of reads and writes.  The essential factors are that more
+ than \constant{_PC_PIPE_BUF} bites are being written by one process in
+ a blocking fashion, while the other process is reading from the other
+ process, also in a blocking fashion.
+ 
+ There are several ways to deal with this situation.
+ 
+ The simplest application change, in many cases, will be to follow this
+ model in the parent process:
+ 
+ \begin{verbatim}
+ import popen2
+ 
+ r, w, e = popen2.popen3('python slave.py')
+ e.readlines()
+ r.readlines()
+ r.close()
+ e.close()
+ w.close()
+ \end{verbatim}
+ 
+ with code like this in the child:
+ 
+ \begin{verbatim}
+ import os
+ import sys
+ 
+ # note that each of these print statements
+ # writes a single long string
+ 
+ print >>sys.stderr, 400 * 'this is a test\n'
+ os.close(sys.stderr.fileno())
+ print >>sys.stdout, 400 * 'this is another test\n'
+ \end{verbatim}
+ 
+ In particular, note that \code{sys.stderr} must be closed after
+ writing all data, or \method{readlines()} won't return.  Also note
+ that \function{os.close()} must be used, as \code{sys.stderr.close()}
+ won't close \code{stderr} (otherwise assigning to \code{sys.stderr}
+ will silently close it, so no further errors can be printed).
+ 
+ Applications which need to support a more general approach should
+ integrate I/O over pipes with their \function{select()} loops, or use
+ separate threads to read each of the individual files provided by
+ whichever \function{popen*()} function or \class{Popen*} class was
+ used.