[Python-Dev] potential argparse problem: bad mix of parse_known_args and prefix matching

Eli Bendersky eliben at gmail.com
Tue Nov 26 18:46:21 CET 2013

On Tue, Nov 26, 2013 at 9:38 AM, Guido van Rossum <guido at python.org> wrote:

> I think matching on the shortest unique prefix is common for command line
> parsers in general, not just argparse. I believe optparse did this too, and
> even the venerable getopt does! I think all this originated in the original
> (non-Python) GNU standard for long option parsing. All that probably
> explains why the docs hardly touch upon it.
> As to why parse_known_args also does this, I can see the reasoning behind
> this behavior: to the end user, "--sync" is a valid option, so it would be
> surprising if it didn't get recognized under certain conditions.
> I suppose you were badly bitten by this recently? Can you tell us more
> about what happened?

Sure. We have a Python script that serves as a gateway to another program.
That other program has a "--sync" option. The gateway script has a
"--sync-foo" option. When the gateway script is invoked with "--sync", we'd
expect it to pass it to the program; instead, it matches it to its own
"--sync-foo" and consumes the option.

Practically, this means a big caveat on exactly the use case
parse_known_args was designed for: whenever I have a Python script using
argparse and passing unknown arguments to other programs, I have to
manually make sure there are no common prefixes between any commands to
avoid this problem.

Frankly I don't see how the current behavior can be seen as the intended


> On Tue, Nov 26, 2013 at 9:30 AM, Eli Bendersky <eliben at gmail.com> wrote:
>> Hello,
>> argparse does prefix matching as long as there are no conflicts. For
>> example:
>> argparser = argparse.ArgumentParser()
>> argparser.add_argument('--sync-foo', action='store_true')
>> args = argparser.parse_args()
>> If I pass "--sync" to this script, it recognizes it as "--sync-foo". This
>> behavior is quite surprising although I can see the motivation for it. At
>> the very least it should be much more explicitly documented (AFAICS it's
>> barely mentioned in the docs).
>> If there's another argument registered, say "--sync-bar" the above will
>> fail due to a conflict.
>> Now comes the nasty part. When using "parse_known_args" instead of
>> "parse_args", the above happens too - --sync is recognized for --sync-foo
>> and captured by the parser. But this is wrong! The whole idea of
>> parse_known_args is to parse the known args, leaving unknowns alone. This
>> prefix matching harms more than it helps here because maybe the program
>> we're actually acting as a front-end for (and hence using parse_known_args)
>> knows about --sync and wants to get it.
>> Unless I'm missing something, this is a bug. But I'm also not sure
>> whether we can do anything about it at this point, as existing code *may*
>> be relying on it. The right thing to do would be to disable this prefix
>> matching when parse_known_args is called.
>> Again, at the very least this should be documented (for parse_known_args
>> not less than a warning box, IMHO).
>> Eli
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> --
> --Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131126/89bb2e2e/attachment.html>

More information about the Python-Dev mailing list