
A little thought I have had a few times now, so it is time to pass it on for comment. Basically _every_ time I use getopt, I write code like this: import getopt try: opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:") except getopt.error, why: print why print usage % (os.path.basename(sys.argv[0],)) sys.exit(1) Every single time. I have never used getopt without this code. How about we put this functionality into getopt itself? It could be triggered either by adding a new "usage" param defaulting to None, or by adding a new entry point. ie: opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:", usage=usage) or opts, args = getopt.getoptex(sys.argv[1:], "slc:d:e:", usage=usage) I know it is fairly trivial, but IMO is such a useful module and the pattern is used so regularly that is seems to make sense to add it. Any thoughts? If it is seen favourably, how should we spell it? Mark.

Mark Hammond wrote:
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ? -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Basically _every_ time I use getopt, I write code like this:
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ?
Because _every_ time I use getopt, I write code like that :-) A higher level interface would maybe be handy, but it simply occurred to me that every time I used the module I used that pattern. I just got sick of typing it all the time and wondered if it struck a chord with anyone else (and I dont have or use a general purpose "mhutil" module :-) Im really just trying to save myself 10 lines of boilerplate coding, not introduce a new standard module :-) Mark.

Mark Hammond wrote:
Just a thought :-) I wrote the CommandLine.py for pretty much the same reason: I have quite a few command line apps lying in my bin/ dir and they all did some kind of getopt/sys.argv tricks to handle the input... way to confusing and not easy to maintain. So I decided to take an OO-approach to have them use the same interface with nice help output and to reduce the coding effort. As an example taken from mxDateTime: #!/usr/local/bin/python -u """ Simple Forking Alarm Sample Application for DateTime types and CommandLine. Only works on OSes which support os.fork(). Author: Marc-Andre Lemburg, mailto:mal@lemburg.com """ import time,sys,os from mx.DateTime import * from CommandLine import Application,ArgumentOption class Alarm(Application): header = "Simple Forking Alarm" options = [ArgumentOption('-s', 'set the alarm to now + arg seconds'), ArgumentOption('-m', 'set the alarm to now + arg minutes'), ArgumentOption('-a', 'set the alarm to ring at arg (hh:mm)'), ] version = '0.1' def main(self): atime = now() + (self.values['-s'] or self.values['-m'] * 60 or self.values['-h'] * 3600) * oneSecond abs = self.values['-a'] if abs: atime = strptime(abs,'%H:%M',today(second=0)) if atime < now(): print 'Alarm time has expired...' return print 'Alarm will ring at',atime if not os.fork(): time.sleep((atime - now()).seconds) alarm() os._exit(0) def alarm(): """ Ring alarm """ for i in range(10): sys.stdout.write('\007') sys.stdout.flush() time.sleep(0.2) if __name__ == '__main__': Alarm() Here's the help output this produces: /home/lemburg> alarm -h ------------------------------------------------------------------------ Simple Forking Alarm ------------------------------------------------------------------------ Synopsis: alarm [option] files... Options and default settings: -s arg set the alarm to now + arg seconds -m arg set the alarm to now + arg minutes -a arg set the alarm to ring at arg (hh:mm) -h show this help text --help show this help text --copyright show copyright --examples show examples of usage Version: 0.1 -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[Mark]
Basically _every_ time I use getopt, I write code like this:
[Marc-Andre]
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ?
[Mark]
Because _every_ time I use getopt, I write code like that :-)
[Marc-Andre]
I wrote the CommandLine.py for pretty much the same reason:
Marc-Andre, you're not hearing what Mark is saying. He wants a change to the standard library, and he knows that small additions to existing modules there stand a better chance of adoption than new modules. I personally liked the idea of getoptex() best, except I would call it getopt_or_die(). If the usage message is omitted it can synthesize one from the (short and long) options arguments and sys.argv[0] (the latter being a bit controversial, but it's just a default). Hmm... Perhaps getopt_or_die() shouldn't take the args argument but extract sys.argv[1:] itself? --Guido van Rossum (home page: http://www.python.org/~guido/)

On 30 September 1999, Guido van Rossum said:
Gasp! Guido hath drunk from the poisoned well of the Great Camel and it doth infest his thinking! "or die" indeed -- not an idiom I've seen since the last time I wrote some Perl code (umm, last week?).
Whatever we call it, it should be able to take an explicit argument list, and only default to sys.argv[1:]. What if I want to parse options from an environment variable or a config file? I also like the "don't clobber sys.argv, but return the modified version instead" model -- it's nice to keep a pristine copy of the original argument list! Another problem with getopt is that it doesn't correlate long and short options. I wrote distutils.fancy_getopt (download your copy today! hurry, don't delay -- at this price, it WON'T LAST LONG!) to address this, and to maybe someday do something with help text. On the other hand, don't listen to me -- I tend to write mammoth, bloated, all-singing, all-dancing command-line parsing modules for every new language I encounter. They get more insane with each iteration. I have yet to top my Getopt::Tabular for Perl, though; see http://search.cpan.org/doc/GWARD/Getopt-Tabular-0.3/Tabular.pod if you've ever wondered how far this sort of thing can be taken in a high-level, dynamically typed language. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

I can sympathize with Mark, I have nearly the same code in every script I write. I once wrote a nice (IMO) class for doing all the common things I do with c.l. args, but I can't seem to dig it up at the moment. The idea was basically to have a base class that had all the machinery, while derived classes included specially named methods that did the app-specific option handling. It knew whether the method took 1 or zero arguments (not including self), glommed up the shortarg string and longarg list (with appropriate `:' and `=' thrown in), then parsed the args, dispatching to the handlers, e.g.: class MyOptions(getopt.Options): def handle_a(self): self.alpha = 1 handle_alpha = handle_a def handle_b(self, arg): self.beta = arg handle_beta = handle_b def handle_i(self, arg): try: self.integer = int(arg) except ValueError: self.usage() handle_integer = handle_i and could be used like so: #! /usr/bin/env python # # ... opts = MyOptions(sys.argv[1:], usage=__doc__ % globals()) if opts.alpha: do_my_alpha_thang() if opts.integer = 1: do_something_else() for file in opts.leftoverargs: process_file(file) While I liked this a lot, I seem to remember showing it to Guido, receiving the expected scoffing. So now, I just cut and paste the opt parsing stuff into every script I write :) I think it might be nice to add such a base class to the standard getopt module. -Barry

OK - general agreement. Wheee :-) How about this: def getopt_or_die(opts, long_opts=[], usage=None, prog_name=None, args=None, exit_code = 1): if usage is None, we build a very simple usage string from opts/long_opts. If prog_name is None, we use os.path.basename(sys.argv[0]). This is printed just before the usage message. If args is None, we use sys.argv[1:] exit_code specifies the param to sys.exit() on invalid options. I have based the ordering on my guess at the most likely to be used - but maybe "prog_name" and "args" should be reversed? This should _not_ be all-singing, all-dancing, as it is simple to use the existing getopt.getopt() directly for more esoteric requirements; the old 80-20 rule applies here :-) If the general agreement continues, I will then knock together an implementation. Mark.

On 01 October 1999, Mark Hammond said:
I still think it would be very desirable to tie the short and long options together. Eg. options = [('verbose', 'v'), ('quiet', 'q'), ('thingy', None), (None, 'x') ('output=', 'o:')] opts, args = getopt_or_die (options, usage, ...) Then opts would have possible keys 'verbose', 'quiet', 'thingy', 'x', and 'value' -- never 'v', 'q', or 'o' (Look, I restrained my tendency to invent type systems and auto-generate help text. There may be hope for me yet.) Greg

Greg writes:
Im not convinced this is worth it. I only use "long options" when I have too many, or a few obscure ones. I have never have "-v" synonymous for "--verbose" - why bother? I know I would never type the later:-) The existing mechanism still handles this quite well - the standard "if opt==blah:" simply becomes "if opt in [...]:" - no real drag. Plus its less change for reasonable reward - handy enough I may actually add command-line handling as I create each little test/util script :-) What say anyone else? Go with my "little change", Gregs "only slightly more change" or "don't worry about it"? Mark.

Mark> Im not convinced this [pairing up long and short options] is worth Mark> it. The primary reason where it would be helpful is to generate clearer default usage strings. Aside from that, I'm not sure the extra structure would be worth it. Skip

Guido van Rossum wrote:
Oh, I did get the idea... just wanted to plug my module here in a take-it-or-leave-it way ;-) I usually put such things into my lib/ dir for Python to find -- no need to make them a standard.
Better not: it's sometimes very useful to call the main(args) function of a script in interactive mode which then passes the args list to getopt(). How about adding something like: def getoptex(...,args=None,helptext='Read the source, Luke ;-)'): if args is None: args = sys.argv[1:] ... -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

as reported on slashdot: http://www.perl.com/pub/1999/09/topaz.html (topaz is also the code name for our "rewrite Tkinter in Python", but that's another story ;-) </F>

"FL" == Fredrik Lundh <fredrik@pythonware.com> writes:
FL> as reported on slashdot: FL> http://www.perl.com/pub/1999/09/topaz.html FL> (topaz is also the code name for our "rewrite FL> Tkinter in Python", but that's another story ;-) Here's an interesting quote: When I was trying to figure out how to be persuasive on this subject, I finally realized that Perl may be competing with Java in the problem space, but when you're writing Perl, implementing the Perl runtime, really what you're doing is something equivalent to writing a JVM. You're writing the equivalent of a Java Virtual Machine. Now, would you write a JVM in Eiffel? I don't think so. No, so neither would you write the Perl runtime in Java or in Eiffel. In the context of Python, I disagree that it competes against Java; Python makes a nice complement to Java. And I obviously also think it makes perfect sense to write (a) Python runtime in Java, as JimH has so effectively proven. Another interesting tidbit we've addressed here: Was the syntax appropriate for declaring variables to give appropriate hints to a hypothetical compiler? That is to say MY INT $X or MY STR $Y -- and I thought that the INT and the STR and the NUM should be suffixes, something like MY $X:NUM-and, in fact, that suffix syntax is something that Larry officially has blessed, but just not for this purpose. -Barry

On Thu, 30 Sep 1999, Barry A. Warsaw wrote:
What struck me about that paragraph is that if I understand the comment correctly, Larry is choosing a mapping from a specific syntax to some purpose (unspecified in the excerpt) while an otherwise well-informed Perl user (Chip) 'naturally' wanted to map said syntax to a different semantics. When this happens in Python-land, Guido says "nope, neither". --david

Mark Hammond wrote:
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ? -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

Basically _every_ time I use getopt, I write code like this:
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ?
Because _every_ time I use getopt, I write code like that :-) A higher level interface would maybe be handy, but it simply occurred to me that every time I used the module I used that pattern. I just got sick of typing it all the time and wondered if it struck a chord with anyone else (and I dont have or use a general purpose "mhutil" module :-) Im really just trying to save myself 10 lines of boilerplate coding, not introduce a new standard module :-) Mark.

Mark Hammond wrote:
Just a thought :-) I wrote the CommandLine.py for pretty much the same reason: I have quite a few command line apps lying in my bin/ dir and they all did some kind of getopt/sys.argv tricks to handle the input... way to confusing and not easy to maintain. So I decided to take an OO-approach to have them use the same interface with nice help output and to reduce the coding effort. As an example taken from mxDateTime: #!/usr/local/bin/python -u """ Simple Forking Alarm Sample Application for DateTime types and CommandLine. Only works on OSes which support os.fork(). Author: Marc-Andre Lemburg, mailto:mal@lemburg.com """ import time,sys,os from mx.DateTime import * from CommandLine import Application,ArgumentOption class Alarm(Application): header = "Simple Forking Alarm" options = [ArgumentOption('-s', 'set the alarm to now + arg seconds'), ArgumentOption('-m', 'set the alarm to now + arg minutes'), ArgumentOption('-a', 'set the alarm to ring at arg (hh:mm)'), ] version = '0.1' def main(self): atime = now() + (self.values['-s'] or self.values['-m'] * 60 or self.values['-h'] * 3600) * oneSecond abs = self.values['-a'] if abs: atime = strptime(abs,'%H:%M',today(second=0)) if atime < now(): print 'Alarm time has expired...' return print 'Alarm will ring at',atime if not os.fork(): time.sleep((atime - now()).seconds) alarm() os._exit(0) def alarm(): """ Ring alarm """ for i in range(10): sys.stdout.write('\007') sys.stdout.flush() time.sleep(0.2) if __name__ == '__main__': Alarm() Here's the help output this produces: /home/lemburg> alarm -h ------------------------------------------------------------------------ Simple Forking Alarm ------------------------------------------------------------------------ Synopsis: alarm [option] files... Options and default settings: -s arg set the alarm to now + arg seconds -m arg set the alarm to now + arg minutes -a arg set the alarm to ring at arg (hh:mm) -h show this help text --help show this help text --copyright show copyright --examples show examples of usage Version: 0.1 -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

[Mark]
Basically _every_ time I use getopt, I write code like this:
[Marc-Andre]
Why not just add a higher level interface ? Something like CommandLine.py which is included in mxDateTime ?
[Mark]
Because _every_ time I use getopt, I write code like that :-)
[Marc-Andre]
I wrote the CommandLine.py for pretty much the same reason:
Marc-Andre, you're not hearing what Mark is saying. He wants a change to the standard library, and he knows that small additions to existing modules there stand a better chance of adoption than new modules. I personally liked the idea of getoptex() best, except I would call it getopt_or_die(). If the usage message is omitted it can synthesize one from the (short and long) options arguments and sys.argv[0] (the latter being a bit controversial, but it's just a default). Hmm... Perhaps getopt_or_die() shouldn't take the args argument but extract sys.argv[1:] itself? --Guido van Rossum (home page: http://www.python.org/~guido/)

On 30 September 1999, Guido van Rossum said:
Gasp! Guido hath drunk from the poisoned well of the Great Camel and it doth infest his thinking! "or die" indeed -- not an idiom I've seen since the last time I wrote some Perl code (umm, last week?).
Whatever we call it, it should be able to take an explicit argument list, and only default to sys.argv[1:]. What if I want to parse options from an environment variable or a config file? I also like the "don't clobber sys.argv, but return the modified version instead" model -- it's nice to keep a pristine copy of the original argument list! Another problem with getopt is that it doesn't correlate long and short options. I wrote distutils.fancy_getopt (download your copy today! hurry, don't delay -- at this price, it WON'T LAST LONG!) to address this, and to maybe someday do something with help text. On the other hand, don't listen to me -- I tend to write mammoth, bloated, all-singing, all-dancing command-line parsing modules for every new language I encounter. They get more insane with each iteration. I have yet to top my Getopt::Tabular for Perl, though; see http://search.cpan.org/doc/GWARD/Getopt-Tabular-0.3/Tabular.pod if you've ever wondered how far this sort of thing can be taken in a high-level, dynamically typed language. Greg -- Greg Ward - software developer gward@cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

I can sympathize with Mark, I have nearly the same code in every script I write. I once wrote a nice (IMO) class for doing all the common things I do with c.l. args, but I can't seem to dig it up at the moment. The idea was basically to have a base class that had all the machinery, while derived classes included specially named methods that did the app-specific option handling. It knew whether the method took 1 or zero arguments (not including self), glommed up the shortarg string and longarg list (with appropriate `:' and `=' thrown in), then parsed the args, dispatching to the handlers, e.g.: class MyOptions(getopt.Options): def handle_a(self): self.alpha = 1 handle_alpha = handle_a def handle_b(self, arg): self.beta = arg handle_beta = handle_b def handle_i(self, arg): try: self.integer = int(arg) except ValueError: self.usage() handle_integer = handle_i and could be used like so: #! /usr/bin/env python # # ... opts = MyOptions(sys.argv[1:], usage=__doc__ % globals()) if opts.alpha: do_my_alpha_thang() if opts.integer = 1: do_something_else() for file in opts.leftoverargs: process_file(file) While I liked this a lot, I seem to remember showing it to Guido, receiving the expected scoffing. So now, I just cut and paste the opt parsing stuff into every script I write :) I think it might be nice to add such a base class to the standard getopt module. -Barry

OK - general agreement. Wheee :-) How about this: def getopt_or_die(opts, long_opts=[], usage=None, prog_name=None, args=None, exit_code = 1): if usage is None, we build a very simple usage string from opts/long_opts. If prog_name is None, we use os.path.basename(sys.argv[0]). This is printed just before the usage message. If args is None, we use sys.argv[1:] exit_code specifies the param to sys.exit() on invalid options. I have based the ordering on my guess at the most likely to be used - but maybe "prog_name" and "args" should be reversed? This should _not_ be all-singing, all-dancing, as it is simple to use the existing getopt.getopt() directly for more esoteric requirements; the old 80-20 rule applies here :-) If the general agreement continues, I will then knock together an implementation. Mark.

On 01 October 1999, Mark Hammond said:
I still think it would be very desirable to tie the short and long options together. Eg. options = [('verbose', 'v'), ('quiet', 'q'), ('thingy', None), (None, 'x') ('output=', 'o:')] opts, args = getopt_or_die (options, usage, ...) Then opts would have possible keys 'verbose', 'quiet', 'thingy', 'x', and 'value' -- never 'v', 'q', or 'o' (Look, I restrained my tendency to invent type systems and auto-generate help text. There may be hope for me yet.) Greg

Greg writes:
Im not convinced this is worth it. I only use "long options" when I have too many, or a few obscure ones. I have never have "-v" synonymous for "--verbose" - why bother? I know I would never type the later:-) The existing mechanism still handles this quite well - the standard "if opt==blah:" simply becomes "if opt in [...]:" - no real drag. Plus its less change for reasonable reward - handy enough I may actually add command-line handling as I create each little test/util script :-) What say anyone else? Go with my "little change", Gregs "only slightly more change" or "don't worry about it"? Mark.

Mark> Im not convinced this [pairing up long and short options] is worth Mark> it. The primary reason where it would be helpful is to generate clearer default usage strings. Aside from that, I'm not sure the extra structure would be worth it. Skip

Guido van Rossum wrote:
Oh, I did get the idea... just wanted to plug my module here in a take-it-or-leave-it way ;-) I usually put such things into my lib/ dir for Python to find -- no need to make them a standard.
Better not: it's sometimes very useful to call the main(args) function of a script in interactive mode which then passes the args list to getopt(). How about adding something like: def getoptex(...,args=None,helptext='Read the source, Luke ;-)'): if args is None: args = sys.argv[1:] ... -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 92 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

as reported on slashdot: http://www.perl.com/pub/1999/09/topaz.html (topaz is also the code name for our "rewrite Tkinter in Python", but that's another story ;-) </F>

"FL" == Fredrik Lundh <fredrik@pythonware.com> writes:
FL> as reported on slashdot: FL> http://www.perl.com/pub/1999/09/topaz.html FL> (topaz is also the code name for our "rewrite FL> Tkinter in Python", but that's another story ;-) Here's an interesting quote: When I was trying to figure out how to be persuasive on this subject, I finally realized that Perl may be competing with Java in the problem space, but when you're writing Perl, implementing the Perl runtime, really what you're doing is something equivalent to writing a JVM. You're writing the equivalent of a Java Virtual Machine. Now, would you write a JVM in Eiffel? I don't think so. No, so neither would you write the Perl runtime in Java or in Eiffel. In the context of Python, I disagree that it competes against Java; Python makes a nice complement to Java. And I obviously also think it makes perfect sense to write (a) Python runtime in Java, as JimH has so effectively proven. Another interesting tidbit we've addressed here: Was the syntax appropriate for declaring variables to give appropriate hints to a hypothetical compiler? That is to say MY INT $X or MY STR $Y -- and I thought that the INT and the STR and the NUM should be suffixes, something like MY $X:NUM-and, in fact, that suffix syntax is something that Larry officially has blessed, but just not for this purpose. -Barry

On Thu, 30 Sep 1999, Barry A. Warsaw wrote:
What struck me about that paragraph is that if I understand the comment correctly, Larry is choosing a mapping from a specific syntax to some purpose (unspecified in the excerpt) while an otherwise well-informed Perl user (Chip) 'naturally' wanted to map said syntax to a different semantics. When this happens in Python-land, Guido says "nope, neither". --david
participants (8)
-
Barry A. Warsaw
-
David Ascher
-
Fredrik Lundh
-
Greg Ward
-
Guido van Rossum
-
M.-A. Lemburg
-
Mark Hammond
-
Skip Montanaro