argparse: mutually inclusive arguments
Hi All, I've been using the argparse library for a long time and one use case that repeatedly shows us is the need to have two arguments appear together i.e either both appear or none of them appear. I'll refer to these as *mutually inclusive* in contrast to the existing *mutual exclusive* feature present. I might be wrong but there seems to be no way to enforce this. Of course it is possible to have two arguments that are *required* but it's not always the case that they will be needed. A more relevant example would be a situation in which you have a set of mutually exclusive arguments, one of which is a group of arguments. For example, consider a scenario in which a user is required to enter multiple related files that share the same file root (excluding the extension). The program would then infer the various files (co-files) by searching for the extensions. The first option would have the user only need to provide the file name root. However, the user may also provide the paths to the various co-files. In this case, the co-files must occur together but must occur mutually-exclusively from the file root option. Another example would be a case in which activating one set of options implies the need for other options to be simultaneously activated and for which defaults may not be easily discerned. Quite often the author has to perform pythonic gymnastics to catch such invalid options, which might be better handled by the argparse library. In evaluating the need for this I've come across the following responses: https://stackoverflow.com/questions/15935092/creating-mutually-inclusive-pos... https://bugs.python.org/issue11588 https://bugs.python.org/issue23298 https://stackoverflow.com/questions/19414060/argparse-required-argument-y-if... https://www.debugcn.com/en/article/57242307.html https://picocli.info/#_mutually_dependent_options I perceive there is a compelling use-case to be addressed here despite an earlier request being turned down. With kind regards, *Paul*
On Wed, Feb 24, 2021 at 8:39 AM Paul Korir <polariseke@gmail.com> wrote:
Shooting from the hip here, but this seems like exactly the kind of situation in which the new match-case syntax would be very useful.
--- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Wed, Feb 24, 2021 at 1:38 PM Paul Korir <polariseke@gmail.com> wrote:
I'm of mixed feelings about this. I can absolutely see that it can be useful, and many tools will have this kind of argument requirement. On the other hand, as soon as you mention this pattern, I can think of a number of others that could be useful (and not implausible) that will remain unavailable. So it's a trade-off about how much is done in argparse configuration itself, versus how much is done in manual validation of arguments. Right now, of course, I can write a manual check: if (args.foo and not args.bar) or (args.bar and not args.foo): parser.error("Args foo and bar must occur together") I don't care whether this checking is done with the future match/case construct versus current if/elif style. Obviously, there are possible ways that extra arguments to .add_argument() could describe that check, if such were added. But let us think of other possible patterns. What if we need "at least 2 of these 5 arguments"? I can conceive of ways to add more parameters to .add_arguments() to cover that, but it's not obvious that would be worthwhile. There will probably remain situations where a manual post-validation will be needed. I'm not sure that "mutually inclusive" shouldn't remain as such. Now a personal confession. I'm far too lax about argument requirements in what I write; my audience is either myself or fellow developers. Pretty much I just rely on "you should use this tool correctly. So if hypothetically, the real usage of my tool is: mytool ((-foo & -bar) | (-baz & -bam)) I'll probably just make them all optional and just fail downstream (usually with some message to STDERR, but not in the argument parsing itself). If my tool were widely used by non-developers or semi-developers, more rigor would be desirable. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David Mertz writes:
Perhaps we could provide a more general facility for validation of the parsed argument values. Of course this would be less convenient for the specific case of arguments that must be specified together with no other requirements, but I wonder how often that case arises. In many cases there are other conditions, for example in specifying an interval, args.lower < args.upper (or <= if you'll accept a degenerate interval). Thing is, this "mutual inclusion" condition isn't really about parsing (ie, syntax)[1]; this is about semantics -- like all input validation. I don't object to having it in argparse, but it's going to be so domain-dependent that I don't think we should do more than provide hooks to validate individual arguments (eg, a 0 <= p <= 1 constraint on probabilities or proportions), and an args.validate hook to do cross-argument validation. Steve Footnotes: [1] I guess we could enforce it as syntax by requiring --foo fval --bar bval in that order. But that's not normally how we think of parsing a command line, and I don't think we want the command line to be a full-blown general parser.
Oscar Benjamin writes:
On Fri, 26 Feb 2021 at 02:49, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
"Quis custodiat ipsos custodes?" That is, it has no extra value in validating the args, but by providing a standard place to put validation code, it helps us to validate the validators. For most of my scripts, I'm not going to write *any* command line validation code, but for programs called by other tools, I think it would be helpful to me to have all command line validation in one standard place. We could even have a tool in argparse to report which arguments have validators and which don't, though that smells like overkill (and I myself would probably write it as a search in Emacs Lisp rather than use an argparse-provided tool).
On Wed, Feb 24, 2021 at 8:39 AM Paul Korir <polariseke@gmail.com> wrote:
Shooting from the hip here, but this seems like exactly the kind of situation in which the new match-case syntax would be very useful.
--- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler
On Wed, Feb 24, 2021 at 1:38 PM Paul Korir <polariseke@gmail.com> wrote:
I'm of mixed feelings about this. I can absolutely see that it can be useful, and many tools will have this kind of argument requirement. On the other hand, as soon as you mention this pattern, I can think of a number of others that could be useful (and not implausible) that will remain unavailable. So it's a trade-off about how much is done in argparse configuration itself, versus how much is done in manual validation of arguments. Right now, of course, I can write a manual check: if (args.foo and not args.bar) or (args.bar and not args.foo): parser.error("Args foo and bar must occur together") I don't care whether this checking is done with the future match/case construct versus current if/elif style. Obviously, there are possible ways that extra arguments to .add_argument() could describe that check, if such were added. But let us think of other possible patterns. What if we need "at least 2 of these 5 arguments"? I can conceive of ways to add more parameters to .add_arguments() to cover that, but it's not obvious that would be worthwhile. There will probably remain situations where a manual post-validation will be needed. I'm not sure that "mutually inclusive" shouldn't remain as such. Now a personal confession. I'm far too lax about argument requirements in what I write; my audience is either myself or fellow developers. Pretty much I just rely on "you should use this tool correctly. So if hypothetically, the real usage of my tool is: mytool ((-foo & -bar) | (-baz & -bam)) I'll probably just make them all optional and just fail downstream (usually with some message to STDERR, but not in the argument parsing itself). If my tool were widely used by non-developers or semi-developers, more rigor would be desirable. -- The dead increasingly dominate and strangle both the living and the not-yet born. Vampiric capital and undead corporate persons abuse the lives and control the thoughts of homo faber. Ideas, once born, become abortifacients against new conceptions.
David Mertz writes:
Perhaps we could provide a more general facility for validation of the parsed argument values. Of course this would be less convenient for the specific case of arguments that must be specified together with no other requirements, but I wonder how often that case arises. In many cases there are other conditions, for example in specifying an interval, args.lower < args.upper (or <= if you'll accept a degenerate interval). Thing is, this "mutual inclusion" condition isn't really about parsing (ie, syntax)[1]; this is about semantics -- like all input validation. I don't object to having it in argparse, but it's going to be so domain-dependent that I don't think we should do more than provide hooks to validate individual arguments (eg, a 0 <= p <= 1 constraint on probabilities or proportions), and an args.validate hook to do cross-argument validation. Steve Footnotes: [1] I guess we could enforce it as syntax by requiring --foo fval --bar bval in that order. But that's not normally how we think of parsing a command line, and I don't think we want the command line to be a full-blown general parser.
Oscar Benjamin writes:
On Fri, 26 Feb 2021 at 02:49, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
"Quis custodiat ipsos custodes?" That is, it has no extra value in validating the args, but by providing a standard place to put validation code, it helps us to validate the validators. For most of my scripts, I'm not going to write *any* command line validation code, but for programs called by other tools, I think it would be helpful to me to have all command line validation in one standard place. We could even have a tool in argparse to report which arguments have validators and which don't, though that smells like overkill (and I myself would probably write it as a search in Emacs Lisp rather than use an argparse-provided tool).
participants (5)
-
David Mertz -
Oscar Benjamin -
Paul Korir -
Ricky Teachey -
Stephen J. Turnbull