Author: brett.cannon Date: Sun Sep 27 21:42:40 2009 New Revision: 75097 Log: Add PEP 389: argparse - new command line parsing module. Added: peps/trunk/pep-0389.txt (contents, props changed) Added: peps/trunk/pep-0389.txt ============================================================================== --- (empty file) +++ peps/trunk/pep-0389.txt Sun Sep 27 21:42:40 2009 @@ -0,0 +1,224 @@ +PEP: 389 +Title: argparse - new command line parsing module +Version: $Revision$ +Last-Modified: $Date$ +Author: Steven Bethard <steven.bethard@gmail.com> +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 25-Sep-2009 +Python-Version: 2.7 and 3.2 +Post-History: + + +Abstract +======== +This PEP proposes inclusion of the argparse [1]_ module in the Python +standard library in Python 2.7 and 3.2. + + +Motivation +========== +The argparse module is a command line parsing library which provides +more functionality than the existing command line parsing modules in +the standard library, getopt [2]_ and optparse [3]_. It includes +support for positional arguments (not just options), subcommands, +required options, options syntaxes like "/f" and "+rgb", zero-or-more +and one-or-more style arguments, and many other features the other +two lack. + +The argparse module is also already a popular third-party replacement +for these modules. It is used in projects like IPython (the Scipy +Python shell) [4]_, is included in Debian testing and unstable [5]_, +and since 2007 has had various requests for its inclusion in the +standard library [6]_ [7]_ [8]_. This popularity suggests it may be +a valuable addition to the Python libraries. + + +Why aren't getopt and optparse enough? +====================================== +One argument against adding argparse is that thare are "already two +different option parsing modules in the standard library" [9]_. The +following is a list of features provided by argparse but not present +in getopt or optparse: + +* While it is true there are two *option* parsing libraries, there + are no full command line parsing libraries -- both getopt and + optparse support only options and have no support for positional + arguments. The argparse module handles both, and as a result, is + able to generate better help messages, avoiding redundancies like + the ``usage=`` string usually required by optparse. + +* The argparse module values practicality over purity. Thus, argparse + allows required options and customization of which characters are + used to identify options, while optparse explicitly states "the + phrase 'required option' is self-contradictory" and that the option + syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file`` + "are not supported by optparse, and they never will be". + +* The argparse module allows options to accept a variable number of + arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The + optparse module provides an untested recipe for some part of this + functionality [10]_ but admits that "things get hairy when you want + an option to take a variable number of arguments." + +* The argparse module supports subcommands, where a main command + line parser dispatches to other command line parsers depending on + the command line arguments. This is a common pattern in command + line interfaces, e.g. ``svn co`` and ``svn up``. + + +Why isn't the functionality just being added to optparse? +========================================================= +Clearly all the above features offer improvements over what is +available through optparse. A reasonable question then is why these +features are not simply provided as patches to optparse, instead of +introducing an entirely new module. In fact, the original development +of argparse intended to do just that, but because of various fairly +constraining design decisions of optparse, this wasn't really +possible. Some of the problems included: + +* The optparse module exposes the internals of its parsing algorithm. + In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed + to be available to callbacks [11]_. This makes it extremely + difficult to improve the parsing algorithm as was necessary in + argparse for proper handling of positional arguments and variable + length arguments. For example, ``nargs='+'`` in argparse is matched + using regular expressions and thus has no notion of things like + ``parser.largs``. + +* The optparse extension APIs are extremely complex. For example, + just to use a simple custom string-to-object conversion function, + you have to subclass ``Option``, hack class attributes, and then + specify your custom option type to the parser, like this:: + + class MyOption(Option): + TYPES = Option.TYPES + ("mytype",) + TYPE_CHECKER = copy(Option.TYPE_CHECKER) + TYPE_CHECKER["mytype"] = check_mytype + parser = optparse.OptionParser(option_class=MyOption) + parser.add_option("-m", type="mytype") + + For comparison, argparse simply allows conversion functions to be + used as ``type=`` arguments directly, e.g.:: + + parser = argparse.ArgumentParser() + parser.add_option("-m", type=check_mytype) + + But given the baroque customization APIs of optparse, it is unclear + how such a feature should interact with those APIs, and it is + quite possible that introducing the simple argparse API would break + existing custom Option code. + +* Both optparse and argparse parse command line arguments and assign + them as attributes to an object returned by ``parse_args``. + However, the optparse module guarantees that the ``take_action`` + method of custom actions will always be passed a ``values`` object + which provides an ``ensure_value`` method [12]_, while the argparse + module allows attributes to be assigned to any object, e.g.:: + + foo_object = ... + parser.parse_args(namespace=foo_object) + foo_object.some_attribute_parsed_from_command_line + + Modifying optparse to allow any object to be passed in would be + difficult because simply passing the ``foo_object`` around instead + of a ``Values`` instance will break existing custom actions that + depend on the ``ensure_value`` method. + +Because of issues like these, which made it unreasonably difficult +for argparse to stay compatible with the optparse APIs, argparse was +developed as an independent module. Given these issues, merging all +the argparse features into optparse with no backwards +incompatibilities seems unlikely. + + +Deprecation of getopt and optparse +================================== +There is still some debate over the best way (if at all) to encourage +users to move from getopt and optparse to their replacement, +argparse. The current recommendation of this PEP is the following +conservative deprecation strategy: + +* Python 3.2, Python 2.7 and any later Python 2.X releases -- + PendingDeprecation warnings, which by default are not displayed, + and documentation notes directing users of getopt and optparse to + argparse. + +* Python 3.3 -- Same as above + +* Python 3.4 -- Deprecation warnings for getopt and optparse, which + by default *are* displayed. + +Though this is slower than the usual deprecation process, it seems +prudent to avoid producing any casually visible Deprecation warnings +until Python 3.X has had some additional time to attract developers. + + +Open Issues +=========== +The argparse module supports Python from 2.3 up through 3.2 and as a +result relies on traditional ``%(foo)s`` style string formatting. It +has been suggested that it might be better to use the new style +``{foo}`` string formatting [13]_. This seems like a good idea, but +would break backwards compatibility for existing argparse-based +scripts unless we can come up with a way to reasonably support both +syntaxes. + + +References +========== +.. [1] argparse + (http://code.google.com/p/argparse/) + +.. [2] getopt + (http://docs.python.org/library/getopt.html) + +.. [3] optparse + (http://docs.python.org/library/optparse.html) + +.. [4] argparse in IPython + (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html) + +.. [5] argparse in Debian + (http://packages.debian.org/search?keywords=python-argparse) + +.. [6] 2007-01-03 request for argparse in the standard library + (http://mail.python.org/pipermail/python-list/2007-January/592646.html) + +.. [7] 2009-06-09 request for argparse in the standard library + (http://bugs.python.org/issue6247) + +.. [8] 2009-09-10 request for argparse in the standard library + (http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html) + +.. [9] Fredrik Lundh response to [6]_ + (http://mail.python.org/pipermail/python-list/2007-January/592675.html) + +.. [10] optparse variable args + (http://docs.python.org/library/optparse.html#callback-example-6-variable-arg...) + +.. [11] parser.largs and parser.rargs + (http://docs.python.org/library/optparse.html#how-callbacks-are-called) + +.. [12] take_action values argument + (http://docs.python.org/library/optparse.html#adding-new-actions) + +.. [13] use {}-formatting instead of %-formatting + (http://bugs.python.org/msg89279) + + +Copyright +========= +This document has been placed in the public domain. + + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 + End: