[Chicago] capturing output from subprocesses
Noel Thomas Taylor
nttaylor at uchicago.edu
Thu Nov 10 18:07:56 CET 2005
Thanks for this great example. I've been experimenting with it, and it
could be the answer to my prayers. One question about forking: I know that
when you do a fork() call, the code gets duplicated in memory, the child
gets its own pid as the return value of the fork, and the parent gets
But how much code gets duplicated, and can a single fork call
significantly impact your memory? In your "child_process.py" for
example, does the whole module get duplicated? If this function were
just one in a giant file thousands of lines long, would that whole file
get duplicated? If your code has a call to fork() in it, does that mean
you should isolate it into a smaller module which you then import, or does
that make a difference?
Or maybe the duplication is virtual and the two processes are really
occupying the same memory space?
I'm sorry I can't make the meeting tonight.
On Tue, 8 Nov 2005, Jess Balint wrote:
> I made a prototype you can use. It's a simple combination of creating
> pipe()s for stdin and stderr, then dup2 them into the streams after a
> fork(), before an exec(). I've attach the python and a test shell
> script (that kills itself). You should be able to see it capture the
> output. (If there is a problem with the attachments, I will put them
> on a web site or something.)
> On 11/8/05, Ian Bicking <ianb at colorstudy.com> wrote:
>> Noel Thomas Taylor wrote:
>>> Hi Ian,
>>> I could try that, but in the case of the real application whose output I
>>> want to capture, I have no control over how much output it produces.
>> I thought it would be an interesting test to understand exactly what is
>> going on, even if it isn't exactly what you are expecting to receive.
>>> Do you have any thoughts about recapturing the output of an aborted child
>>> process before the memory that is buffering its output gets blown away?
>> Since it is OS buffering, shouldn't the OS handle that for you? I don't
>> know; I would suggest starting with a test, then finding something that
>> passes that test.
>> Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org
>> Chicago mailing list
>> Chicago at python.org
More information about the Chicago