subprocess woes

[Mike Driscoll]

On Sep 15, 2:26 pm, kj <no.em... at please.post> wrote:
> I'm trying to write a function, sort_data, that takes as argument
> the path to a file, and sorts it in place, leaving the last "sentinel"
> line in its original position (i.e. at the end).  Here's what I
> have (omitting most error-checking code):
>
> def sort_data(path, sentinel='.\n'):
>     tmp_fd, tmp = tempfile.mkstemp()
>     out = os.fdopen(tmp_fd, 'wb')
>     cmd = ['/usr/local/bin/sort', '-t', '\t', '-k1,1', '-k2,2']
>     p = Popen(cmd, stdin=PIPE, stdout=out)
>     in_ = file(path, 'r')
>     while True:
>         line = in_.next()
>         if line != sentinel:
>             p.stdin.write(line)
>         else:
>             break
>     in_.close()
>     p.stdin.close()
>     retcode = p.wait()
>     if retcode != 0:
>         raise CalledProcessError(retcode, cmd)
>     out.write(sentinel)
>     out.close()
>     shutil.move(tmp, path)
>
> This works OK, except that it does not catch the stderr from the
> called sort process.  The problem is how to do this.  I want to to
> avoid having to create a new file just to capture this stderr
> output.  I would like instead to capture it to an in-memory buffer.
> Therefore I tried using a StringIO object as the stderr parameter
> to Popen, but this resulted in the error "StringIO instance has no
> attribute 'fileno'".
>
> How can I capture stderr in the scenario depicted above?
>
> TIA!
>
> kynn

According to the docs for subprocess module (which you don't appear to
be using even though that's what you used for your subject line), you
can set stderr to stdout:

http://docs.python.org/library/subprocess.html

You can use cStringIO to create a "file-like" object in memory:

http://docs.python.org/library/stringio.html

-------------------
Mike Driscoll

Blog:   http://blog.pythonlibrary.org