[ python-Bugs-1056441 ] subprocess fails for args="...", executable="..."

[SourceForge.net]

Bugs item #1056441, was opened at 2004-10-29 00:32
Message generated for change (Comment added) made by astrand
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1056441&group_id=5470

Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Russell Owen (reowen)
Assigned to: Nobody/Anonymous (nobody)
Summary: subprocess fails for args="...", executable="..."

Initial Comment:
In Python 2.4b1 I am trying to use the subprocess module and 
running into problems when args is a string and executable is 
specified.

For example:

>>> p = subprocess.Popen(
  executable = "xpaget",
  args = "xpaget ds9 mode",
  shell = True,
  stdout = subprocess.PIPE,
)
results in this mysterious error msg:
>>> sh: illegal option -- c
and the data in p.stdout is a boatload of help text
that strongly suggests xpaget never got the "ds9 mode" command.

Piping stdin and stderr make no difference, of course,
but omitting the stdout makes one strange difference:
I don't see the error message, just the boatload of help text.


Removing the executable argument makes it work as expected:
>>> p = subprocess.Popen(
  args = "xpaget ds9 mode",
  shell = True,
  stdout = subprocess.PIPE,
)
>>> p.stdout.read()
'pointer\n'

And the executable argument works fine if I specify the arguments as 
a list and don't use shell=True:
p = subprocess.Popen(
  executable = "xpaget",
  args = ["xpaget", "ds9", "mode"],
  stdout = subprocess.PIPE,
)
>>> p.stdout.read()
'pointer\n'

xpa and ds9 are free from <http://hea-www.harvard.edu/RD/ds9/>
but I hope think they are not needed to debug this.

I saw this problem using a unix installation of Python 2.4b1 on MacOS 
X 10.3.5.

----------------------------------------------------------------------

>Comment By: Peter Åstrand (astrand)
Date: 2004-10-31 07:33

Message:
Logged In: YES 
user_id=344921

reowen: 

>While on that subject, should specifying args as a string
without
>specifying shell=True should produce a warning?

No. This is documented:

"A string will be treated as a sequence with the string as
the only item (the program to execute)."

>Thanks for subprocess. It's a wonderful module.

Thanks :)

----------------------------------------------------------------------

Comment By: Peter Åstrand (astrand)
Date: 2004-10-31 07:30

Message:
Logged In: YES 
user_id=344921

Russell: What did you expect? Shells doesn't support that
you specify an alternative executable, only the raw system
calls do. 

Currently, if you use shell=True, the executable argument
can be used to specify an alternative shell, if you are not
happy with the default /bin/sh. I think this is useful. 

I must admit that this is not very well documented, though. 

----------------------------------------------------------------------

Comment By: Russell Owen (reowen)
Date: 2004-10-29 23:58

Message:
Logged In: YES 
user_id=431773

I looked at the code and the problem is here:

        def _execute_child(self, args, executable, preexec_fn, close_fds,
                           cwd, env, universal_newlines,
                           startupinfo, creationflags, shell,
                           p2cread, p2cwrite,
                           c2pread, c2pwrite,
                           errread, errwrite):
            """Execute program (POSIX version)"""

            if isinstance(args, types.StringTypes):
                args = [args]

            if shell:
                args = ["/bin/sh", "-c"] + args

            if executable == None:
                executable = args[0]

You can se that if shell is true (as it must be if one specifies args as a string) 
and executable is supplied, one ends p with a mess, i.e. the executable ends 
up trying to execute  ["/bin/sh", "-c", args]

Fortunately cwd offers a hackaround. Still, I hope this can be either fixed or 
else can produce a warning.

While on that subject, should specifying args as a string without specifying 
shell=True should produce a warning?

Thanks for subprocess. It's a wonderful module. I am really looking forward 
to using it (once a few kinks get worked out).

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1056441&group_id=5470