Python deadlock using subprocess.popen and communicate

Roy Smith roy at
Wed Sep 21 23:32:19 EDT 2011

In article 
<098f3d78-85f5-44e7-ba72-f2270a24d1b0 at>,
 Atherun <atherun at> wrote:

> This is on windows with python 2.6.
> I can't seem to remove a possibility of a  deadlock in one of my
> scripts at the moment.  Its not a constant deadlock but it appears
> from time to time.  The code is below:
> try:
>         process =
> subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
>         NotDone = True
>         data = []
>         while NotDone:
>             NotDone = False
>             result = process.poll()
>             if result == None:
>                 NotDone = True
>             if NotDone:
>                 out, err = process.communicate()
>                 Log(out)
> I was able to get the program to write out the stack trace for my
> threads and it is usually deadlocked on or
> _internal_poll of subproceess.
> I've even tried the above, with using "with
> tempfile.NamedTemporaryFiles() as buff:" and writing to the file, but
> it still deadlocks.  In the past I work around this by running fewer
> processes asynchronously but I would really like to get this solved so
> I don't have to wait to see if it'll be caused with any changes I
> make.
> Any tips would be appreciated.

My reading of the docs 
ts) says that Popen.poll() doesn't return a value, it sets the object's 
return code attribute, which you can then interrogate.

More than that, your loop logic looks amazingly complicated for what's 
basically a simple thing.  I suggest something along the lines of:

   # assuming process.returncode is initially None 
   while process.returncode is None:
      out, err = process.communicate()

I haven't tested that, but I think (from reading the docs) that's the 
right idea.

More information about the Python-list mailing list