subprocess.Popen question

[Thomas Rachel]

Am 16.08.2011 09:03 schrieb Danny Wong (dannwong):
> Hi All,
> 	I'm executing a command which I want to capture the
> standard/stderr output into a file (which I have with the code below),
> but I also want the standard output to go into a variable so I can
> process the information for the next command. Any ideas? Thanks.
>
> CMD_OUTPUT = subprocess.Popen(COMMAND, stdout=PROJECT_LOGFILE,
> stderr=subprocess.STDOUT)

This way it won't work - but if you do

   sp = subprocess.Popen(COMMAND, stdout=subprocess.PIPE, 
stderr=subprocess.PIPE)

you can flexibly work with both of them. But then the two streams can 
become intermixed.

After that, you would have to emulate the select() behaviour of 
communicate(), but act differently on them. If you get data on stdout, 
you capture and append to the file, for data on stderr you only append.

Maybe this could work for you:

      def communicate2(self, input):
          """ input is readable, output is a generator which yields data
          """
          read_set = []
          write_set = []
          stdout = None # Return
          stderr = None # Return

          # Translate newlines, if requested.  We cannot let the file
          # object do the translation: It is based on stdio, which is
          # impossible to combine with select (unless forcing no
          # buffering).
          if self.universal_newlines and hasattr(file, 'newlines'):
              xlate=lambda s:self._translate_newlines(s)
          else:
              xlate=lambda s:s
          if self.stdin:
              # Flush stdio buffer.  This might block, if the user has
              # been writing to .stdin in an uncontrolled fashion.
              self.stdin.flush()
          if self.stdout:
              read_set.append(self.stdout)
              stdout = []
          if self.stderr:
              read_set.append(self.stderr)
              stderr = []

          inpcrsr=0 # der ist neu...
          rest=''
          eofseen=False
          while read_set or write_set:
              rlist, wlist, xlist = select.select(read_set, write_set, [])

              if self.stdin in wlist:
                  # When select has indicated that the file is writable,
                  # we can write up to PIPE_BUF bytes without risk
                  # blocking.  POSIX defines PIPE_BUF >= 512
                  #pipebuf=512
                  pipebuf=os.fpathconf(self.stdin.fileno(),'PC_PIPE_BUF')
                  if not eofseen: buf=input.read(pipebuf-len(rest)) # 
else it stays ''
                  if not buf: eofseen=True
                  if rest or buf:
                      bytes_written = os.write(self.stdin.fileno(), 
rest+buf)
                      if bytes_written > len(rest):
                              rest=buf[bytes_written-len(rest):]
                      else:
                              rest=rest[bytes_written:]+buf
                  else:
                      self.stdin.close()
                      write_set.remove(self.stdin)

              if self.stdout in rlist:
                  data = os.read(self.stdout.fileno(), 1024)
                  if data == "":
                      self.stdout.close()
                      read_set.remove(self.stdout)
                  #stdout.append(data)
                  #yield stdout,data
                  yield 1,xlate(data)

              if self.stderr in rlist:
                  data = os.read(self.stderr.fileno(), 1024)
                  if data == "":
                      self.stderr.close()
                      read_set.remove(self.stderr)
                  #stderr.append(data)
                  yield 2,xlate(data)
          self.wait()

(Im not sure if it is to 100% correct.)

This yields tuples (1, <data>) for stdout data and tuples (2, <data>) 
for stderr data.