[Python-Dev] test_pty.py hangs in verbose mode on Mac OS X?

[Jean-Paul Calderone]

On Fri, 13 Apr 2007 10:32:28 -0400, Barry Warsaw <barry at python.org> wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I've been getting some test failures in Python 2.5 svn head on Mac OS
>X 10.4.9 which I'm not getting on Linux (Ubuntu feisty beta).
>test_sqlite and test_zipimport both fail, however, when run in
>verbose mode (e.g. ./python.exe Lib/test/test_sqlite.py) both pass.
>
>But that's not exactly why I'm writing this email <wink>.  In the
>course of trying to debug this, I ran the following on my Mac:
>
>make TESTOPTS=-v test
>
>This runs the entire test suite in verbose mode, and you do get a lot
>of output.  However the test suite hangs on test_pty.py.  In fact, if
>you run that test alone:
>
>./python.exe Lib/test/test_pty.py
>
>it too hangs for me.  The reason is that in verbose mode, debug()
>actually prints stuff to stdout and on the Mac, when the child of the
>pty.fork() writes to its stdout, it blocks and so the parent's waitpid
>() never returns.  This doesn't happen on Linux though; the child's
>stdout prints don't block, it exits, and the parent continues after
>the waitpid().
>
>Here's a very simple program that reproduces the problem:
>
>- -----snip snip-----
>import os, pty, sys
>
>pid, fd = pty.fork()
>print >> sys.stderr, pid, fd
>if pid:
>     os.waitpid(pid, 0)
>else:
>     os._exit(0)
>- -----snip snip-----
>
>stderr, stdout, doesn't matter.  This hangs on the Mac but completes
>successfully on Linux.  Of course, in neither case do you see the
>child's output.
>
>I don't know if this is caused by a bug in the Mac's pty
>implementation or something we're doing wrong on that platform.  I
>played around with several modifications to pty.fork() on the Mac,
>including letting it drop down to the openpty()/os.fork() code, even
>adding an explicit ioctl(slave_fd, TIOCSCTTY) call which Stevens
>chapter 19 recommends for 4.3+BSD. I can't get it to not block.

What about reading from the child in the parent before calling waitpid?

>
>Barring a fix to pty.fork() (or possibly os.forkpty()) for the Mac,
>then I would like to at least make test_pty.py not block when run in
>verbose mode.  A very simple hack would add something like this to
>the "if pid == pty.CHILD" stanza: "def debug(msg): pass", possibly
>protected by a "if verbose:".  A less icky hack would be to read the
>output from the master_fd in the parent, though you have to be
>careful with that on Linux else the read can throw an input/output
>error.
>
>Disabling debug output is band-aid yes, and any application on the
>Mac like the above snippet will still fail.  If anybody has any
>suggestions, I'm all ears, but I've reached the limit of my pty-fu.
>

I don't think this is an OS X PTY bug.  Writing to a blocking file
descriptor can block.  Programs that do this need to account for the
possibility.

Jean-Paul