Direct interaction with subprocess - the curse of blocking I/O
chambon.pascal at gmail.com
Mon Jun 29 15:15:52 EDT 2009
I've had real issues with subprocesses recently : from a python script,
on windows, I wanted to "give control" to a command line utility, i.e
forward user in put to it and display its output on console. It seems
simple, but I ran into walls :
- subprocess.communicate() only deals with a forecast input, not
step-by-step user interaction
- pexpect module is unix-only, and for automation, not interactive input
- when wanting to do all the job manually (transfering data between the
standard streams of the python program and the binary subprocess, I met
the issue : select() works only on windows, and python's I/O are
blocking, so I can't just, for example, get data from the subprocess'
stdout and expect the function to return if no input is present - the
requesting thread might instead block forever.
Browsing the web, I found some hints :
- use the advanced win32 api to create non-blocking I/O : rather
complicated, non portable and far from the python normal files
- use threads that block on the different streams and eat/feed them
without ever stopping : rather portable, but gives problems on shutdown
(How to terminate these threads without danger ? On some OSes, a process
never dies as long as any thread - even "daemonic" - lives, I've seen
people complaining about it).
So well, I'd like to know, do you people know any solution to this
simple problem - making a user interact directly with a subprocess ? Or
would this really require a library handling each case separately (win32
api, select().....) ?
Thanks a lot for your interest and advice,
More information about the Python-list