Do subprocess.PIPE and subprocess.STDOUT sametime
jak
nospam at please.ty
Tue May 9 17:22:34 EDT 2023
Horst Koiner ha scritto:
> Hi @all,
> i'm running a program which is still in development with subprocess.run (Python version 3.10), further i need to capture the output of the program in a python variable. The program itself runs about 2 minutes, but it can also freeze in case of new bugs.
>
> For production i run the program with stdout=subprocess.PIPE and i can fetch than the output later. For just testing if the program works, i run with stdout=subprocess.STDOUT and I see all program output on the console, but my program afterwards crashes since there is nothing captured in the python variable. So I think I need to have the functionality of subprocess.PIPE and subprcess.STDOUT sametime.
>
> What I tried until now:
> 1. Poll the the output and use Popen instead:
>
> # Start the subprocess
> process = subprocess.Popen(['./test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>
> captured_output = b''
> process_running = True
> while process_running:
> process_running = (process.poll() is not None)
> for pipe in [ process.stdout, process.stderr ]:
> while line := pipe.readline():
> print(line)
> captured_output += line
>
> print(captured_output)
> return_code = process.returncode
>
> => But this is discouraged by the python doc, since it says that polling this way is prone to deadlocks. Instead it proposes the use of the communicate() function.
>
> 2. Use communicate() with timeout.
> => This works not at all since when the timeout occurs an exception is thrown and communicate returns at all.
>
> 3. Use threading instead
> => For being that simple and universal like subprocess you will more or less reimplement subprocess with threading, like its done in subprocess.py. Just for a debug output the effort is much to high.
>
> #######################################################
> Do you have further ideas for implementing such a behavior?
> Do you think that a feature request should be done of I'm omitting something obvious?
>
> Thanks you in advance for your suggestions,
> Horst.
>
I agree with @'thomas Passin' but I solved in a different way, I made
the Readline() not blocking. even if I believe his idea better than
mine:
os.set_blocking(process.stdout.fileno(), False)
os.set_blocking(process.stderr.fileno(), False)
More information about the Python-list
mailing list