[Python-Dev] Issue 15906; regression in argparse in Python 3.3, 3.2, and 2.7

[Terry Reedy]

On 9/11/2012 11:34 AM, Barry Warsaw wrote:
> Issue 15906 describes a problem with argparse that is breaking lots of code in
> Ubuntu.  This is a recent regression caused by the fix for issue 12776, and it
> affects Python 2.7, 3.2, and 3.3.
>
> I posted a diff that should fix the problem, but at the heart of it is a
> semantic ambiguity in argparse that needs clarification.  This needs to be
> cleared up before a proper patch can be applied.  I have submitted a patch for
> what *I* think reasonable semantics should be, but let's see what you think.
>
> Issue 12776 tried to fix a problem illustrated by this example:
>
> -----snip snip-----
> import argparse
>
> parser = argparse.ArgumentParser()
> parser.add_argument('--file', type=open, default='/etc/passwd')
> args = parser.parse_args()
>
> print(args.file.read())
>
> args = parser.parse_args(['--file', '/etc/group'])
> print(args.file.read())
> -----snip snip-----
>
> What this code is (IMO, sensibly) trying to do is say that args.file will
> always be an open file, regardless of whether the path comes from the default
> value or is given on the command line.
>
> The problem is that this breaks some other, also sensible code:
>
> -----snip snip-----
> import argparse
>
> parser = argparse.ArgumentParser()
> parser.add_argument("--test", dest="test", type=str,
>      default=[], action='append')
>
> args = parser.parse_args(['--test', 'bar'])
> args.test.append('baz')
>
> args = parser.parse_args()
> args.test.append('baz')
> -----snip snip-----
>
> This code is saying, I want to accumulate string arguments into a list,
> regardless of whether any arguments are given on the command line.

This second example strikes me (naively, as English speaker but not 
argparse user) as 'wrong' in that 'default' is being misused to mean 
'start value that is always used to generate the final value' [as in 
sum(iterable, start=0)], rather than 'final value that is only used if 
nothing else is given' (as in nearly all uses of 'default' in Python). 
Perhaps this is what you meant by "semantic ambiguity".

As I see it, storing is done *with* a default or explicit value, 
appending is done *to* a start value *with* whatever. Perhaps reusing 
'default' instead of using a new name such as 'start' was a bit too 
clever for our own good ;-).

> The fix for issue 12776 broke the last two lines of the second example,
> because in the no-command-line-arguments-given case, arg.test is the *string*
> "[]" and not the actual empty list object.

This seems even more wrong (as in slightly crazy) as it switches the 
meaning of 'default' within one parser example rather than between 
parser examples.

> It seems to me that the semantics could reasonably be implied to mean that the
> type converter should only be applied to the default value when
> action='store', as is the default.  Then in the second example, because
> action='append', the type conversion would not be applied (it makes no sense
> to do so).

I arrive at the same conclusion, I believe, by saying that for a given 
parser, the type converter should always or never be applied to 
'default', which should mean converting or not when the parser is 
created. Append to 'default as base or start' should mean not converting.

-- 
Terry Jan Reedy