[Chicago] capturing output from subprocesses

[Jess Balint]

On 11/14/05, Noel Thomas Taylor <nttaylor at uchicago.edu> wrote:
> concept. The idea is that the child process should be producing output
> fairly regularly. If some number of seconds go by during which it does not
> produce output, the parent should decide that the child is stuck and
> should kill it. Adding the timeout to the select call makes this possible.

Ok, that makes fine sense now. :)

> A couple of questions: I notice you don't import the pty module anywhere,
> nor do you use os.forkpty(). I guess just opening the file in
> "/dev/pty..." makes it a pseudo-terminal? Isn't this a little dangerous? I
> would have thought the pty module would make things safer all around, but
> maybe it's not necessary at all?

This is the standard way to open a pty. As far as the module goes, it
doesn't seem to have support on Solaris. The pty module only has the
"select" call and "os" doesn't have the "openpty" call. I don't
consider this dangerous at all. If you have some other evidence, I'd
be interested to hear it.

> Also, the code you sent unfortunately doesn't work under Irix. On that OS,
> the child's termination signal will never be picked up, and it will run
> indefinitely. The timeout in the select call is ineffective.

What do you mean the child's termination signal will not be picked up?
Are you saying the SIGCHLD isn't sent to the parent process? Or isn't
received for some reason? If there is a difference in the select
implementation, we can probably work around that with a timer, but I
think that's probably how they implement it... The other thing is if
the pty isn't closed correctly, we might use the exception fd's from
select to know when it's done. Either way, I'll have to take a closer
look at it.

> I mentioned that I'd been using Noah Spurrier's pexpect module, which
> contains a lot of the functionality I'm looking for. Originally, pexpect
> had the exact same problem under Irix that your code does, but I got in
> touch with Noah and he made a hack for it.
>
> In his email to me he wrote:
>    "Irix is a weird platform. It seems to have no reliable way to test if
>     a child process is alive or not without blocking... I managed to find
>     a hack for this, but it adds a 2 second timeout to every call to
>     read_nonblocking(). That means you can't have a timeout less than 3
>     seconds. That's probably not a big deal in general."

I'm not familiar with all the Python aspects of this or how
read_nonblocking is implemented. Maybe kill(pid, 0) will work? As I
said before maybe the exc. fd set will help here.

> I orignally sent this email two days ago with a copy of "pexpect.py"
> attached to it, but it got held up by the list administrator for being too
> large, so rather than wait to see if it goes through, I'm canceling that
> original mail and sending this one instead, which has a link to the code
> instead of the code itself:

I see what he's doing, but I don't have know the exact issue he is
trying to work around.

> If you have time, please visit the above site and take a look. There's a
> good bit of code there, but I wonder if you might look over the part that
> specifically addresses the Irix hack (just search for the word 'irix' in
> the source). My ultimate goal is to understand exactly what's going on,
> and I think I stand a better chance with your code, which is much shorter
> and more focused on this one specific problem, than with pexpect, which is
> meant to be a much more general. If I could eventually make my own program
> work with a short piece of code that I throroughly understand, that is
> preferable to using the pexpect module, which is more of a black box.
>
> And the short-term goal is to incorporate Noah's Irix hack into the code
> that you sent me. What do you think? I'm working on getting you a guest
> account on our Irix box for testing, in case you don't have one available.

Hopefully you have a C compiler on there too. ;)

Jess