On Mon, May 24, 2010 at 11:23 AM, Bill Janssen wrote:

C. Titus Brown wrote:

...

On Mon, May 24, 2010 at 07:46:32AM -0700, Bill Janssen wrote:

...

I'm kind of surprised to see that the subprocess module doesn't have a documented function for calling to properly quote command-line arguments. Nor does it seem to do so automatically -- at least on Unix.

Bill, I may be about to reveal my profound ignorance here, but...

do you need to quote arguments passed as a list on UNIX? My impression was that if shell=False, then it wasn't needed.

Yes, that's right. You need to quote them only if you are passing a whole command line to be evaluated as a shell, as with os.system() or shell=True and args a string with subprocess. I think subprocess gets all this more or less right.

In my case, I was building up a command line to pass as a single string to subprocess, and needed a way to quote the args explicitly. That's why I went looking at this.

I've had to do this sort of thing for similar reasons, like calling remote ssh commands, where a shell command is embedded in a positional argument. I implemented my own thing, but the way pipes.quote works would have been nicer (the implementation could use a couple regexes instead of iterating over strings though). -- Ian Bicking | http://blog.ianbicking.org

Éric Araujo wrote:

...

In my case, I was building up a command line to pass as a single string to subprocess, and needed a way to quote the args explicitly. That's why I went looking at this. FYI, if you need the reverse operation (string → list), look at shlex.

Keep in mind that these routines typically only work on Unix-like systems, where the shell has a set of well defined rules for quoting and parsing. On Windows, each executable is in charge of parsing its own parameters from a string. This is usually (but certainly not always) done by the C runtime startup. But different runtimes have different rules. Sadly, you sometimes have to know which executable you're calling before you can know how to escape its arguments. -- Eric.

On May 25, 2010, at 2:54 PM, Paul Moore wrote:

On 25 May 2010 13:22, Eric Smith wrote:

...

Keep in mind that these routines typically only work on Unix-like systems, where the shell has a set of well defined rules for quoting and parsing. On Windows, each executable is in charge of parsing its own parameters from a string. This is usually (but certainly not always) done by the C runtime startup. But different runtimes have different rules. Sadly, you sometimes have to know which executable you're calling before you can know how to escape its arguments.

However, on Windows the rules used by the C runtime are a pretty good baseline to work to. That's what subprocess does, and it's a good practical solution. It's not perfect, but practicality beats purity and all that....

I'd say that this is a classic case where a library encapsulating cross-platform differences [1] is extremely useful. If the stdlib contained a cross-platform shell-quoting module, I'm pretty sure it could replace a big chunk of adhoc application code.

I'd also like to see both uses, splitting and joining posix/windows command lines cleaned up into a separate module. The interface to shlex can be a little daunting to deal with, with it's odd non-posix mode that isn't actually for the Windows shell. Plus it's complicated by the fact that it's also intended to be used as a generic parser. There's already a ticket requesting the ability to parse a Windows command line: http://bugs.python.org/issue1724822 -- Philip Jenvey