Help with pipes, buffering and pseudoterminals
cs at zip.com.au
Tue Apr 7 07:48:03 CEST 2015
On 06Apr2015 21:13, Daniel Ellis <ellisd23 at gmail.com> wrote:
>Wow, thank you for the long and detailed reply.
>As for the question ptys and C, no, I have no experience in that area.
No worries. The use is the same in Python.
BTW, you may want a fixed width font; there are some ASCII diagrams in this
>I've read a bit on the master/slave stuff, but I'm still finding it
>confusing. The manpage for pty states that it creates a "pair" of virtual
>devices, one master and one slave. It doesn't seem like these are simple
>file descriptors for stdin, stdout, and stderr, given that there is only
>one for the master and one for the slave, so I'm really not understanding
>what it means for these devices to be created.
The following explaination is long; I do get to master/slave at the bottom. But
I'm covering stdin/stdout/stderr too.
Think about a serial terminal. On one end there is a computer, with wires to
the terminal. Since the computer runs UNIX, access to the terminal is done with
read and write calls. A read returns characters typed at the terminal by a
person and a write sends characters to the terminal which are displayed to the
person terminal <==cable==> computer
If someone is logged in on a physical terminal, on the computer's end there
will be a program acting on what they type. The UNIX command prompt, the shell,
is what one would normally get after logging in; for moderns users this is
When you open() a file (or device) in UNIX you get a file descriptor handed
to you; this is a single integer which is essentially an index into the
process' array of open files. Index 0 is stdin, 1 stdout, 2 stderr, and other
open files get further indices.
It is important to realise an open() creates, internally, a "file handle",
the data structure that tracks use of the file from that specific open()
call. It has information like the I/O mode (read/write/append) and the file
position (where data lands when you write or where data comes from when you
The file descriptor, the index, points into an array of _references_ to file
handles. You can dup() a file descriptor to get another file descriptor i.e.
another index; it will refer to the _same_ file handle internally.
process: fd 0 -> tty handle -> tty device
fd 1 ---^ ^
fd 2 -----+
The operating system arranges the commection of the shell to the terminal. Your
usual program has by default a stdin, stdout and stderr. These are _all_ the
same file handle, duplicated to each of the three file descriptors 0, 1 and 2
respectively. On the operating system side, the OS has performed _one_ open()
call on the terminal device and handed the caller a single file descriptor. The
caller then calls dup() (or modernly, dup2()) to present the open terminal as
stdin, stdout and stderr.
That file handle is open for read and write, as you might imagine.
If you want to see this, type:
lsof -p $$
at your shell prompt. Observe fd 0, 1, 2.
Why the duplication? This lets you do redirection! You can attach something
else to a program's stdout, for example:
ls > files.txt
without disturbing stdin or stderr. And so forth.
A pty behaves the same way, except that there is no physical device involved.
There is just an abstraction which behaves like a physical device:
You see, wire telegraph is a kind of a very, very long cat. You
pull his tail in New York and his head is meowing in Los Angeles.
Do you understand this? And radio operates exactly the same way:
you send signals here, they receive them there. The only difference
is that there is no cat.
- Albert Einstein, when asked to describe radio
The main reason for ptys is that there is a class of programs which present a
terminal interface to users (xterm, sshd, telnetd, etc).
So, something like xterm _draws_ a GUI representation of a terminal for you to
look at, and collects keystrokes (as GUI events), converts them into bytes and
writes them to the pty. For example, it might see "Shift-A-Down" and send code
65 (capital "A"). Similarly it reads from the pty and draws on the screen.
The shell you use in an xterm is on the other side of the pty. The code 65 the
xterm sent above is seen read as code 65 by the shell and treated as a typed
In this scenario, effectively the xterm is a server to the shell: it is
responsible for accepting things from the shell and displaying them, and
providing typed GUI events to the shell as input bytes. It also sets the
initial terminal modes and attaches the shell to the pty.
Correspondingly, when xterm opened a pty for its work the two file descriptors
it gets are "master" and "slave" file descriptors.
The master is kept by xterm and used to read shell output and write to the
The slave is used to set up the shell. The basic scheme is that the xterm
forks. The child xterm instance closes all file descriptors _except_ the slave
descriptor, and then dup()s that descriptor to fd 0, 1 and 2, providing stdin,
stdout and stderr. And the pty is made the child's controlling terminal.
When the (parent) xterm writes to the master descriptor those data appear for
read on the slave file descriptor. Conversely, when the shell writes to its
output (the slave file descriptor) it appears for read on the master descriptor
(for the xterm to read and display).
So the slave descriptor _is_ a perfectly ordinary file descriptor.
Feel free to ask further questions. I'm sure my rambling must occasion some.
Cameron Simpson <cs at zip.com.au>
From a programmer's point of view, the user is a peripheral that types when you
issue a read request. - Peter Williams
More information about the Python-list