Simple process IO capture (Was: "shell-commands" and python!) - process.py (0/1)

[Paul Moore]

On Mon, 24 Sep 2001 13:42:22 -0700, Neil Schemenauer <nas at python.ca> wrote:

>Here's my humble attempt at a popen replacement.  Note that it doesn't
>use the shell (a feature IMHO) and it only works on Unix systems.
>Suggestions for the module name are welcome.  I would like to get
>something like this in the standard library.

I like the idea of having some form of "simplified piping" module in the
standard library. But I would *much* rather see something that is portable (at
least to Unix and Windows - I've no idea if the Mac has the concept of a command
line which would make sense here). The low-level building blocks are of
necessity non-portable, but the higher-level wrappers really ought to be (in
interface, if not implementation). Precisely because the lower levels are
non-portable - a portable higher level interface would make writing portable
programs that bit easier.

A portability requirement makes designing a suitable interface far more
difficult, but I think it's worth it.

(BTW, I could probably write a Win32 implementation of your interface (modulo
the availability of suitable low-level calls - I'd need to depend on either
win32all or dynwin). The only issue would be the argv tuples, which are *not*
the lowest level Win32 form - precisely the reverse of Unix (in Unix, execv()
takes argc/argv, and the shell converts command lines to argv arrays; on Win32,
CreateProcess() takes a command line, and the C runtime for an individual
program splits the command line into an argv array).

I'm relatively happy with the _ChildProcess class as a way of encapsulating the
various FDs relevant to the child. Although I miss the shortcuts my module has
for passing a string as stdin direct, or getting the contents of stdout direct.
Your class' equivalent is more verbose -

    p = execute(...)
    p.write(str)
    p.stdin.close()
    ret = p.read()
    p.stdout.close()

I might simply add extra "convenience" methods,

    def input(self, str):
        self.stdin.write(str)
        self.stdin.close()
        return self # to allow chaining, see below
    def output(self):
        str = self.stdout.read()
        self.stdout.close()
        return str

which allows

    ret = execute(...).input(str).output()

Hmm, no that still feels messy. I can't get away from the fact that my version

    ret = run(..., input=str)

feels cleaner.

And by the way, your version doesn't offer any way of "merging" stdout and
stderr. Not reading both, but merging as in 2>&1.

And I'd *still* like to add a way of specifying a pipeline, with all the
inter-process plumbing handled automatically. Something like

    child_obj = execute(("tr a-z A-Z", "sort", "uniq"))

(reverting to command-string form rather than argv-tuple, to avoid obscuring the
issue with nested tuples...)

OK, that's enough unstructured brain-dumping. I'll think about this some more,
and post a proposal.

[BTW, I assume that avoiding the shell is an important requirement on Unix? It's
less of an obvious thing to do on Win32, where a number of key commands like
DIR, COPY, DEL, RENAME and MOVE are shell built-ins, and not normally available
as separate EXEs]

Paul.