Optparse object containing generators: only seem to work if given particular names?

John O'Hagan research at johnohagan.com
Fri Oct 31 07:48:41 CET 2008

Here's a strange one for you:

I have a generator function which produces lists of numbers and takes options 
which influence the output. The generator contains a loop, and to enable the 
options to have a different value on each iteration, the options may 
themselves be instances of the same generator, in the form of attributes of 
an optparse object. In the body of the loop, the next() method is called on 
each generator to get the new value each time. 

This enables me to either: 

- pass an option either a single value as  and have it yielded repeatedly by a 
dummy generator, 

- or, by using the (arbitrarily chosen) word "CTRL:" after the option, 
followed by another set of options - in quotes - suitable for the main 
generator, to pass a changing value.

The problem: whether it works or not depends on what name is given to the 
destination in parser.values()! Strange but true.

I haven't yet been able to figure out if there's any pattern to this, but 
names that work include: "add", "A", "rand", and "updown". Some that don't 
are "descend", "random", and "Z".

When it doesn't work, the error is:

TypeError: unsupported operand type(s) for +: 'int' and 'generator'

So apparently when given one of the "wrong" names, calling options.
[wrongname].next() yields another generator object, whereas with the "right" 
names, it yields the expected value.

 Below is a working "sandbox" version of the relevant part of the actual 
program, but with a hard-coded value for the number list and only a single 
option, --add, which either adds the number given next on the command line to 
each element of the list, or, if followed by CTRL: '' " (that's a pair of 
empty quotes) will sequentially do the same for each member of the list. (If 
the option is repeated inside the quotes, the altered list will be used to 
apply the addition, and so on.)

If you don't believe me (and I'm finding this hard to believe myself), try 
some different names in the two inline-commented places (of course, both have 
to be the same). Some will work, others won't.

I'm baffled, and hope someone here may be interested in figuring out why this 
is happening.


John O'Hagan

Here's the code:

import sys
from optparse import OptionParser
def my_callback(option, opt_str, value, parser):
	rargs = parser.rargs

	if rargs[0] == "CTRL:" :
		setattr(parser.values, option.dest+"_ctrl", rargs[1].split())	
		value = abs(int(rargs[0]))
		setattr(parser.values, option.dest, value)
def parse(option_list):
	parser = OptionParser()		
	parser.add_option("--add", action="callback", callback=my_callback, 
dest="add")  ## The troublesome name!

	(options, args) = parser.parse_args(option_list)
	return options

def numerical_ctrl(generator):
	for number_list in generator:
		for number in number_list:
			yield number
def dummy_ctrl(value):
	while 1:	
		yield value

#This next bit should ideally be incorporated into the callback function above 
#It converts the attributes of the "options" object to generators.
def iterizer( options ):
	for opt in vars( options )  :
		if "_ctrl" in opt:
			ctrl_opts_list = getattr( options, opt )
			setattr(options, opt, dummy_ctrl( getattr( options, opt ) ) )
			opt = opt.replace( "_ctrl", "" )
			ctrl_opts = parse( ctrl_opts_list )
			ctrl_opts = iterizer( ctrl_opts )
			generator = phrase_engine( ctrl_opts )
			generator = numerical_ctrl( generator )	
			generator = dummy_ctrl( getattr( options, opt ) )

		setattr (options, opt , generator)

	return options

#The number list generator:

def phrase_engine(options):
	counter = 0
	while counter < 10:
		sequence = [1, 2, 3]
		add = options.add.next() ##Here is the troublesome name again.
		if add :
			sequence = [ i + add for i in sequence ]
		yield sequence
		counter += 1
options = parse( sys.argv[ 1: ] )
options = iterizer(options)

for i in phrase_engine(options):
	print  i

More information about the Python-list mailing list