Optparse object containing generators: only seem to work if given particular names?

[Peter Otten]

John O'Hagan wrote:

> Here's a strange one for you:
> 
> I have a generator function which produces lists of numbers and takes
> options which influence the output. The generator contains a loop, and to
> enable the options to have a different value on each iteration, the
> options may themselves be instances of the same generator, in the form of
> attributes of an optparse object. In the body of the loop, the next()
> method is called on each generator to get the new value each time.
> 
> This enables me to either:
> 
> - pass an option either a single value as  and have it yielded repeatedly
> by a dummy generator,
> 
> - or, by using the (arbitrarily chosen) word "CTRL:" after the option,
> followed by another set of options - in quotes - suitable for the main
> generator, to pass a changing value.
> 
> The problem: whether it works or not depends on what name is given to the
> destination in parser.values()! Strange but true.
> 
> I haven't yet been able to figure out if there's any pattern to this, but
> names that work include: "add", "A", "rand", and "updown". Some that don't
> are "descend", "random", and "Z".
> 
> When it doesn't work, the error is:
> 
> TypeError: unsupported operand type(s) for +: 'int' and 'generator'
> 
> So apparently when given one of the "wrong" names, calling options.
> [wrongname].next() yields another generator object, whereas with the
> ["right"
> names, it yields the expected value.
> 
>  Below is a working "sandbox" version of the relevant part of the actual
> program, but with a hard-coded value for the number list and only a single
> option, --add, which either adds the number given next on the command line
> to each element of the list, or, if followed by CTRL: '' " (that's a pair
> of empty quotes) will sequentially do the same for each member of the
> list. (If the option is repeated inside the quotes, the altered list will
> be used to apply the addition, and so on.)
> 
> If you don't believe me (and I'm finding this hard to believe myself), try
> some different names in the two inline-commented places (of course, both
> have to be the same). Some will work, others won't.
> 
> I'm baffled, and hope someone here may be interested in figuring out why
> this is happening.
> 
> Thanks,
> 
> John O'Hagan
> 
> Here's the code:
> 
> import sys
> from optparse import OptionParser
> 
> def my_callback(option, opt_str, value, parser):
> 
> rargs = parser.rargs
> 
> if rargs[0] == "CTRL:" :
> setattr(parser.values, option.dest+"_ctrl", rargs[1].split())
> else:
> value = abs(int(rargs[0]))
> setattr(parser.values, option.dest, value)
> 
> def parse(option_list):
> 
> parser = OptionParser()
> 
> parser.add_option("--add", action="callback", callback=my_callback,
> dest="add")  ## The troublesome name!
> 
> (options, args) = parser.parse_args(option_list)
> return options
> 
> 
> def numerical_ctrl(generator):
> 
> for number_list in generator:
> for number in number_list:
> yield number
> 
> def dummy_ctrl(value):
> while 1:
> yield value
> 
> #This next bit should ideally be incorporated into the callback function
> #above (later!)
> #It converts the attributes of the "options" object to generators.
> 
> def iterizer( options ):
> 
> for opt in vars( options )  :
> 
> if "_ctrl" in opt:
> ctrl_opts_list = getattr( options, opt )
> setattr(options, opt, dummy_ctrl( getattr( options, opt ) ) )
> opt = opt.replace( "_ctrl", "" )
> ctrl_opts = parse( ctrl_opts_list )
> ctrl_opts = iterizer( ctrl_opts )
> generator = phrase_engine( ctrl_opts )
> generator = numerical_ctrl( generator )
> else:
> generator = dummy_ctrl( getattr( options, opt ) )
> 
> setattr (options, opt , generator)
> 
> return options
> 
> #The number list generator:
> 
> def phrase_engine(options):
> 
> counter = 0
> while counter < 10:
> 
> sequence = [1, 2, 3]
> 
> add = options.add.next() ##Here is the troublesome name again.
> 
> if add :
> sequence = [ i + add for i in sequence ]
> 
> yield sequence
> 
> counter += 1
> 
> options = parse( sys.argv[ 1: ] )
> options = iterizer(options)
> 
> for i in phrase_engine(options):
> print  i

Just a quick note: Your iterizer() implementation seems to depend on xxx
occuring before xxx_ctrl. That's only the case for some values:

>>> dict(add_ctrl=1, add=2)
{'add': 2, 'add_ctrl': 1}
>>> dict(random_ctrl=1, random=2)
{'random_ctrl': 1, 'random': 2}

While you can enforce that with sorted(vars(options)) I suggest that you try
to simplify your code a bit -- you know, debugging being twice as hard as
coding...

Peter