subprocess.Popen() output to logging.StreamHandler()

Vinay Sajip vinay_sajip at yahoo.co.uk
Thu Apr 10 19:13:33 CEST 2008


On Apr 10, 1:11 pm, "sven _" <svens... at gmail.com> wrote:
> Version: Python 2.5.1 (r251:54863, Mar  7 2008, 04:10:12)
>
> My goal is to have stdout and stderr written to alogginghandler.
> This code does not work:
>
> # START
> importlogging, subprocess
> ch =logging.StreamHandler()
> ch.setLevel(logging.DEBUG)
> subprocess.call(['ls', '-la'], 0, None, None, ch, ch)
> # END
>
> Traceback (most recent call last):
>   File "log.py", line 5, in <module>
>    subprocess.call(['ls', '-la'], 0, None, None, ch, ch)
>   File "/usr/lib/python2.5/subprocess.py", line 443, in call
>    return Popen(*popenargs, **kwargs).wait()
>   File "/usr/lib/python2.5/subprocess.py", line 586, in __init__
>    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
>   File "/usr/lib/python2.5/subprocess.py", line 941, in _get_handles
>    c2pwrite = stdout.fileno()
> AttributeError: StreamHandler instance has no attribute 'fileno'
>
> This is because subprocess.Popen() expects file descriptors to write
> to, andlogging.StreamHandler() does not supply it. The StreamHandler
> could supply its own stdout file descriptor, but then Popen() would
> write directly to that file, bypassing all theloggingfluff.
>
> A possible solution would be to make a named pipe (os.mkfifo()), have
> Popen() write to that, and then have some horrendous hack run select()
> or similar on the fifo to read from it and finally pass it to
> StreamHandler.
>
> Are there better solutions?
>
> sven

Thomas was almost right, but not quite - you can't call info on a
Handler instance, only on a Logger instance. The following script:

import logging
import subprocess

logging.basicConfig(level=logging.INFO) # will log to stderr of this
script

s = subprocess.Popen( ['ls','-la'], stdout=subprocess.PIPE )
while 1:
    line = s.stdout.readline()
    exitcode = s.poll()
    if (not line) and (exitcode is not None):
	break
    line = line[:-1]
    logging.info("%s", line)

produces the following output:

INFO:root:total 204
INFO:root:drwxr-xr-x 35 vinay vinay 4096 2008-04-10 18:06 .
INFO:root:drwxr-xr-x  3 root  root  4096 2008-03-22 07:09 ..
INFO:root:-rw-------  1 vinay vinay  685 2008-04-10
17:26 .bash_history
INFO:root:-rw-r--r--  1 vinay vinay  220 2008-03-22 07:09 .bash_logout
INFO:root:-rw-r--r--  1 vinay vinay 2327 2008-03-22 07:09 .bashrc
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-05 02:21 .bluefish
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-03-22 07:18 .cache
INFO:root:drwxr-xr-x  5 vinay vinay 4096 2008-03-22 07:32 .config
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Desktop
INFO:root:-rw-------  1 vinay vinay   28 2008-04-10 00:33 .dmrc
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Documents
INFO:root:-rw-------  1 vinay vinay   16 2008-03-22 07:17 .esd_auth
INFO:root:lrwxrwxrwx  1 vinay vinay   26 2008-03-22 07:09 Examples -> /
usr/share/example-content
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-10 00:21 .fontconfig
INFO:root:drwx------  4 vinay vinay 4096 2008-04-10 17:23 .gconf
INFO:root:drwx------  2 vinay vinay 4096 2008-04-10 17:43 .gconfd
INFO:root:-rw-r-----  1 vinay vinay    0 2008-03-24 19:13 .gksu.lock
INFO:root:drwx------  9 vinay vinay 4096 2008-04-10 00:31 .gnome2
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22
07:17 .gnome2_private
INFO:root:drwx------  2 vinay vinay 4096 2008-04-10 00:33 .gnupg
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-10
00:33 .gstreamer-0.10
INFO:root:-rw-r--r--  1 vinay vinay  108 2008-04-10 00:33 .gtk-
bookmarks
INFO:root:dr-x------  2 vinay vinay    0 2008-04-10 00:33 .gvfs
INFO:root:-rw-------  1 vinay vinay  167 2008-04-10
00:33 .ICEauthority
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-03-22 07:18 .local
INFO:root:drwx------  3 vinay vinay 4096 2008-03-22 07:18 .metacity
INFO:root:drwx------  4 vinay vinay 4096 2008-03-24 19:13 .mozilla
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Music
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-10 00:31 .nautilus
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .netbeans
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .netbeans-
registration
INFO:root:drwx------  3 vinay vinay 4096 2008-04-05
02:15 .openoffice.org2
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Pictures
INFO:root:-rw-r--r--  1 vinay vinay  566 2008-03-22 07:09 .profile
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Public
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 08:01 .pulse
INFO:root:-rw-------  1 vinay vinay  256 2008-03-22 07:17 .pulse-
cookie
INFO:root:-rw-r--r--  1 vinay vinay 1973 2008-04-10 18:06 .recently-
used.xbel
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22 07:17 .ssh
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .subversion
INFO:root:-rw-r--r--  1 vinay vinay    0 2008-03-22
07:34 .sudo_as_admin_successful
INFO:root:drwxr-xr-x  4 vinay vinay 4096 2008-03-22 09:10 .sudoku
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Templates
INFO:root:-rw-r--r--  1 vinay vinay  297 2008-04-10 18:06 test.py
INFO:root:-rw-r--r--  1 vinay vinay  291 2008-04-10 18:06 test.py~
INFO:root:-rwxr--r--  1 vinay vinay   75 2008-03-22 07:33 update
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 08:22 .update-
manager-core
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22 07:18 .update-
notifier
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Videos
INFO:root:drwxr-xr-x  7 vinay vinay 4096 2007-10-08 14:56 vmware-tools-
distrib
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-07 00:19 .wapi
INFO:root:-rw-------  1 vinay vinay  121 2008-04-10 00:33 .Xauthority
INFO:root:-rw-r--r--  1 vinay vinay  146 2008-04-09
21:33 .xscreensaver-getimage.cache
INFO:root:-rw-r--r--  1 vinay vinay 5102 2008-04-10 17:31 .xsession-
errors
vinay at zeta-hardy:~$ python test.py | more
INFO:root:total 204
INFO:root:drwxr-xr-x 35 vinay vinay 4096 2008-04-10 18:07 .
INFO:root:drwxr-xr-x  3 root  root  4096 2008-03-22 07:09 ..
INFO:root:-rw-------  1 vinay vinay  685 2008-04-10
17:26 .bash_history
INFO:root:-rw-r--r--  1 vinay vinay  220 2008-03-22 07:09 .bash_logout
INFO:root:-rw-r--r--  1 vinay vinay 2327 2008-03-22 07:09 .bashrc
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-05 02:21 .bluefish
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-03-22 07:18 .cache
INFO:root:drwxr-xr-x  5 vinay vinay 4096 2008-03-22 07:32 .config
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Desktop
INFO:root:-rw-------  1 vinay vinay   28 2008-04-10 00:33 .dmrc
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Documents
INFO:root:-rw-------  1 vinay vinay   16 2008-03-22 07:17 .esd_auth
INFO:root:lrwxrwxrwx  1 vinay vinay   26 2008-03-22 07:09 Examples -> /
usr/share/example-content
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-10 00:21 .fontconfig
INFO:root:drwx------  4 vinay vinay 4096 2008-04-10 17:23 .gconf
INFO:root:drwx------  2 vinay vinay 4096 2008-04-10 17:43 .gconfd
INFO:root:-rw-r-----  1 vinay vinay    0 2008-03-24 19:13 .gksu.lock
INFO:root:drwx------  9 vinay vinay 4096 2008-04-10 00:31 .gnome2
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22
07:17 .gnome2_private
INFO:root:drwx------  2 vinay vinay 4096 2008-04-10 00:33 .gnupg
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-10
00:33 .gstreamer-0.10
INFO:root:-rw-r--r--  1 vinay vinay  108 2008-04-10 00:33 .gtk-
bookmarks
INFO:root:dr-x------  2 vinay vinay    0 2008-04-10 00:33 .gvfs
INFO:root:-rw-------  1 vinay vinay  167 2008-04-10
00:33 .ICEauthority
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-03-22 07:18 .local
INFO:root:drwx------  3 vinay vinay 4096 2008-03-22 07:18 .metacity
INFO:root:drwx------  4 vinay vinay 4096 2008-03-24 19:13 .mozilla
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Music
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-10 00:31 .nautilus
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .netbeans
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .netbeans-
registration
INFO:root:drwx------  3 vinay vinay 4096 2008-04-05
02:15 .openoffice.org2
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Pictures
INFO:root:-rw-r--r--  1 vinay vinay  566 2008-03-22 07:09 .profile
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Public
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 08:01 .pulse
INFO:root:-rw-------  1 vinay vinay  256 2008-03-22 07:17 .pulse-
cookie
INFO:root:-rw-r--r--  1 vinay vinay 1973 2008-04-10 18:07 .recently-
used.xbel
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22 07:17 .ssh
INFO:root:drwxr-xr-x  3 vinay vinay 4096 2008-04-05 02:09 .subversion
INFO:root:-rw-r--r--  1 vinay vinay    0 2008-03-22
07:34 .sudo_as_admin_successful
INFO:root:drwxr-xr-x  4 vinay vinay 4096 2008-03-22 09:10 .sudoku
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Templates
INFO:root:-rw-r--r--  1 vinay vinay  339 2008-04-10 18:07 test.py
INFO:root:-rw-r--r--  1 vinay vinay  303 2008-04-10 18:07 test.py~
INFO:root:-rwxr--r--  1 vinay vinay   75 2008-03-22 07:33 update
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 08:22 .update-
manager-core
INFO:root:drwx------  2 vinay vinay 4096 2008-03-22 07:18 .update-
notifier
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-03-22 07:17 Videos
INFO:root:drwxr-xr-x  7 vinay vinay 4096 2007-10-08 14:56 vmware-tools-
distrib
INFO:root:drwxr-xr-x  2 vinay vinay 4096 2008-04-07 00:19 .wapi
INFO:root:-rw-------  1 vinay vinay  121 2008-04-10 00:33 .Xauthority
INFO:root:-rw-r--r--  1 vinay vinay  146 2008-04-09
21:33 .xscreensaver-getimage.cache
INFO:root:-rw-r--r--  1 vinay vinay 5102 2008-04-10 17:31 .xsession-
errors

You should be able to adapt this to your specific requirement.

Regards,

Vinay Sajip

N.B. This test was run using Python 2.5.2 on Ubuntu Hardy Heron
(beta):

Python 2.5.2 (r252:60911, Apr  8 2008, 21:49:41)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2



More information about the Python-list mailing list