[getopt-sig] option-class

[Greg Ward]

[Albert -- could you please limit your posts to < 80 columns?
 Reformatting them is almost as annoying as trying reading them after
 "> " has been added.]

On 21 February 2002, A.T. Hofkamp said:
> Therefore, I took the example from the Optik page, and re-implemented it in
> argtools.
> Obviously, I am a bit biased to my own argtools package.

Excellent!  I've just started the process of reimplementing parts of
some of my real-life tools using other option parsing libraries.  I was
hoping a few other people would get into the game.

> The Optik example (from its hoem page):
[...]
> Since I have Python 1.5.2, I cannot run this example.

The goal of this SIG is to come up with a new and better option parsing
library for Python 2.3.  Therefore, compatibility with obsolete Python
versions that certain Linux distributors with their heads firmly stuck
in the sand keep distributing is a very minor issue, IMHO.

> The argtools re-implementation:
> -------------------------
> from argtools import *
> 
> parser = OptionParser()
> 
> file    = parser.AddOption(StringOption("f file"))
> verbose = parser.AddOption(  VoidOption("q quiet"))
> 
> parser.ProcessCommandLine()
> args = parser.ArgList()
> -------------------------
> the results can be queried with
> 
> print "Results:"
> print "File option: value="+file.Value()
> print "Quiet option: used="+str(verbose.Used())
> 
> 
> 
> When comparing both pieces of code, I notice the following:
> 
> 1) I can run the argtools example.
>    One can argue that Python 2.0 is now minimal, but it is not standard
>    everywhere.

Again, this is mostly irrelevant since we're trying to come up with a
new getopt for Python 2.3.  If you're stuck with Python 1.5.2, you
certainly won't have the new getopt from 2.3.

> 2) Optik code looks bulkier.

That has improved a bit in the current Optik CVS; the -f/--file option
from the Optik home page can now be spelled:

  parser.add_option("-f", "--file", dest="filename",
                    help="write report to FILE", metavar="FILE")

...and if you don't mind having a filename stored in a variable called
'file', you can drop the 'dest' and 'metavar' arguments too.

>    if I have more "-q"-like options, there is a lot redundancy, which
>    leads to the usual maintenance and consistency problems.

Boolean options are tricky.  Very often, I want two different options
writing to the same variable; the canonical example is -v sets 'verbose'
true and -q sets it false.  You can do this with Optik, but it isn't
terribly pretty:

  parser.add_option("-v", action="store_true", dest="verbose")
  parser.add_option("-q", action="store_false", dest="verbose")

That looks OK, but it does get bulky when you add synynoms (--verbose,
--quiet), help, and a default value.  I'd like to hear how other option
libraries handle this very common case!

>    There are exactly 80 chars on the -q line, exclusive the whitespace at the
>    front, so having a single line for a single option inside a function or
>    class is difficult at least.

I find most option specifications take 2 or 3 lines of code.  That's too
bad, but you get a lot of bang for your buck.  So I'm not hung up on
cramming each option spec into a single line.

> 3) Optik has at most 1 short and 1 long name for an option.

Incorrect.  You can have as many short and long names as you like:

  parser.add_option("-p", "--use-pipes", "--synchronous", ...)

>    In argtools, I solved this by having a single string for all option names
>    (with a short option being a single letter, and a long option being more
>    than 1 letter).

Yuck.  Why put multiple values into a single string when something like

  def add_option (self, *args)

works just as well?  This is quite possibly my favourite trick in Optik;
see OptionParser.add_option() and Option.__init__() in
  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/optik/optik/lib/option_parser.py?rev=1.33&content-type=text/vnd.viewcvs-markup
  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/optik/optik/lib/option.py?rev=1.15&content-type=text/vnd.viewcvs-markup
for how it works.

> 4) Optik puts all results in 'options'. Argtools stores the results in the
>    option itself.

Right: input in one object, output in another.  I don't like the idea of
commingling option specifications (which are fixed by the application
programmer) with option values (which are chosen by the user, vary from
run to run, and are completely transient).  It just feels wrong.  With
the Optik way, you can easily reuse the same Option object for multiple
arg parsing runs; can you do that with Argtools?

Also, if you have 17 different options, it's quite easy to pass them all
around together: just pass the options object.  When you put each option
value into a separate local variable, what then?  You'll probably create
your own options object just to keep them together; why not let the
option parsing library do that for you?

However, I should play with Argtools a bit to see how I like it first.

>    (What if 2 options use the same 'dest' ?!?o

Often a desirable feature, as in -v/-q above.  But it does increase the
semantic load on the programmer.

> (especially, if you have no
> control over those names)).

These are variable names.  Why would you not have control over the
variable names used in your program?

> 5) Extending Optik to new types of options is done by deriving a new option, a
>    new parser, or by using callback functions.
>    Argtools allows extensions with new types of options by deriving a new
>    option class.
>    I think that in a widely used standard Python package, the concept
>    'callback' should not be used, and possibly, it should not even be available,
>    as it defies all forms of good programming.

I'm not keen on callbacks, but sometimes it's easier to write a callback
than to derive your own Option subclass.  An application with a lot of
callbacks would probably be better off with an iterative interface anyways.

>    Especially, in an object-oriented framework, deriving a new class
>    is _THE_ only way to go in my opinion. Anything else may be
>    possible, but not for the normal user.

That really strikes me as overly dogmatic.  Subclassing is a great
extensibility mechanism, and in an OO framework it's almost always the
preferred extensibility mechanism.  But (heresy alert!) there's more
than one way to do it.

>    Optik differentiates between adding a new type of option-argument (e.g. a
>    boolean argument), and a new action that should take place.

Exactly.  That's the key insight that took me *years* to arrive at; when
I had it, I sat down and wrote Optik within a week.  (I have been
pondering this problem for a looong time, and I could never get around
the idea that "list to append to" is a type just like "string" or "int"
is a type.  Thinking in that rut makes it awfully hard to, say, add an
"output directory" type, and then automagically gain the ability to
accumulate a list of output directories.  When I finally figured out
that "append to list" is an *action* like "store to a variable",
everything just came together.)

>    In the former
>    case, a new option should be derived, in the latter case, a new parser
>    should be derived.

Incorrect.  In both cases, you extend the Option class.  See
extending.txt:
  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/optik/optik/extending.txt

>    Also, I need to copy variables, make the right modifications in various
>    places, etc. To me, extending Optik doesn't feel simple and logical.

It's a bit tricky because you have to set various class attributes in
your subclass.  It's either that or define a bunch of methods to
substitute for the class attributes -- eg. "is_typed_action()" rather
than TYPED_ACTIONS -- and then subclasses would have to override those
methods instead.  

>    I don't see the logic behind the changes, other than the fact that the
>    parser seems to be doing stuff it shouldn't do.

Have you read extending.txt?  And what precisely do you mean by "the
parser doing stuff it shouldn't do"?

>    Argtools on the other hand allows all extensions w.r.t. option-types or
>    option processing of arguments to be done inside a (derived) option class.
>    By limiting all changes to a single class, extending seems to be much easier.

OK, I want to add an "output directory" option type.  Can I get
"accumulate a list of output directories" for free?  Or do I have to add
a "list of output directories" option type too?

> I hope I didn't offend people with my plugging of argtools, at least
> it was not my intention.  My goal is not to push argtools as _THE_
> solution. Instead, I hope to have shown that adding the 'option class'
> concept in the final option processing module which we are designing
> seems to make a lot of extensions fairly natural.  Therefore, the
> concept should be carefully considered for inclusion.

I'm not offended, whatever you might think after reading this post.
Looks like there are definitely some good ideas in Argtools, and I'll
give it a good look.

        Greg
-- 
Greg Ward - Linux nerd                                  gward@python.net
http://starship.python.net/~gward/
Don't hate yourself in the morning -- sleep till noon.