[Python-bugs-list] [ python-Bugs-436259 ] exec*/spawn* problem with spaces in args

[noreply@sourceforge.net]

Bugs item #436259, was opened at 2001-06-25 20:17
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=436259&group_id=5470

Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Submitted By: Ben Hutchings (wom-work)
>Assigned to: Tim Peters (tim_one)
Summary: exec*/spawn* problem with spaces in args

Initial Comment:
DOS and Windows processes are not given an argument
vector, as Unix processes are; instead they are given a
command line and are expected to perform any necessary
argument parsing themselves. Each C run-time library
must convert command lines into argument vectors for
the main() function, and if it includes exec* and
spawn* functions then those must convert argument
vectors into a command-line. Naturally, the various
implementations differ in interesting ways.

The Visual C++ run-time library (MSVCRT) implementation
of the exec* and spawn* functions is particularly awful
in that it simply concatenates the strings with spaces
in-between (see source file cenvarg.c), which means
that arguments with embedded spaces are likely to turn
into multiple arguments in the new process. Obviously,
when Python is built using Visual C++, its os.exec* and
os.spawn* functions behave in this way too. MS prefers
to work around this bug (see Knowledge Base article
Q145937) rather than to fix it. Therefore I think
Python must work around it too when built with Visual C++.

I experimented with MSVCRT and Cygwin (using the
attached program print_args.c) and could not find a way
to convert an argument vector into a command line that
they would both convert back to the same argument
vector, but I got close.

MSVCRT's parser requires spaces that are part of an
argument to be enclosed in double-quotes. The
double-quotes do not have to enclose the whole
argument. Literal double-quotes must be escaped by
preceding them with a backslash. If an argument
contains literal backslashes before a literal or
delimiting double-quote, those backslashes must be
escaped by doubling them. If there is an unmatched
enclosing double-quote then the parser behaves as if
there was another double-quote at the end of the line.

Cygwin's parser requires spaces that are part of an
argument to be enclosed in double-quotes. The
double-quotes do not have to enclose the whole
argument. Literal double-quotes may be escaped by
preceding them with a backslash, but then they count as
enclosing double-quote as well, which appears to be a
bug. They may also be escaped by doubling them, in
which case they must be enclosed in double-quotes;
since MSVCRT does not accept this, it's useless. As far
as I can see, literal backslashes before a literal
double-quote must not be escaped and literal
backslashes before an enclosing double-quote *cannot*
be escaped. It's really quite hard to understand what
its rules are for backslashes and double-quotes, and I
think it's broken. If there is an unmatched enclosing
double-quote then the parser behaves as if there was
another double-quote at the end of the line.

Here's a Python version of a partial fix for use in
nt.exec* and nt.spawn*.  This function modifies
argument strings so that the resulting command line
will satisfy programs that use MSVCRT, and programs
that use Cygwin if that's possible.

def escape(arg):
    import re
    # If arg contains no space or double-quote then
    # no escaping is needed.
    if not re.search(r'[ "]', arg):
        return arg
    # Otherwise the argument must be quoted and all
    # double-quotes, preceding backslashes, and
    # trailing backslashes, must be escaped.
    def repl(match):
        if match.group(2):
            return match.group(1) * 2 + '\"'
        else:
            return match.group(1) * 2
    return '"' + re.sub(r'(\*)("|$)', repl, arg) + '"'

This could perhaps be used as a workaround for the
problem. Unfortunately it would conflict with
workarounds implemented at the Python level (which I
have been using for a while).



----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=436259&group_id=5470