[Tutor] How to foreach over a dynamic number of levels

[Kent Johnson]

On Wed, Feb 11, 2009 at 6:20 PM, Isaac Eiland-Hall <daychilde at gmail.com> wrote:

> The group I'm working with has a python program (the Model Framework) that
> runs data through a number of models. For each possible step, there are
> multiple possible models. Each run also is for a single date.
>
> So the ini for Model Framework might be something like this:
>
> step1 = step1model-a
>
> step2 = step2model-a
>
> step3 =  step3model-b
>
> step4 = step4model-a

> The program I'm working on will drive the framework and run it multiple
> times. For example, to run a range of dates; but also, to run a range of
> models – so you might want to run models a-h on days Jan1-Jan30 – that would
> be 8 times 30 = 240 runs.
>
>  The end-user will basically take a Framework ini file, and change
> some values from strings to lists, for example:
>
> step1=["step1model-a","step1model-b","step1model-c"]
>
>
>
> My program will parse that, and know that it must iterate three times
> because of that line alone…
>
>
>
> So I do not know in advance how many keys I will have, nor how many of them
> will be arrays vs. strings, although my initial thought is that I can treat
> strings like single-member arrays – I don't care if I foreach a single time
> – execution time on that order of magnitude is irrelevant…
>
>
>
> I'm really not sure how in the world to accomplish this…
>
>
>
> The best method I can think of would involve somehow iterating through and
> generating code to exec, but frankly I don't even have that as a complete
> working solution… The other thought would be a function that called itself,
> but that is also very very ugly…

It's much easier than that. You haven't asked about reading the ini
file or running the model, so I'll leave that for another time.

Let's assume you have the steps read in to variables like this:
step1=["step1model-a","step1model-b","step1model-c"]
step2 = "step2model-a"
step3 =  ["step3model-a", "step3model-b"]

Make a list containing all the steps:
steps = [step1, step2, step3]

Now you need to convert all steps to lists. This loop builds a new
list from the original step list, wrapping any bare strings in lists:

step_lists = []
for step in steps:
    if isinstance(step, basestring):
        step = [step]
    step_lists.append(step)

Now what you want is the Cartesian product of all the lists. Python
has a function (new in Python 2.6) in the itertools library module
that will do this:

from itertools import product
for model_set in product(*step_lists):
    print model_set

(The * in the argument list means, take the elements of step_list and
use them as the arguments to product())

The output is
('step1model-a', 'step2model-a', 'step3model-a')
('step1model-a', 'step2model-a', 'step3model-b')
('step1model-b', 'step2model-a', 'step3model-a')
('step1model-b', 'step2model-a', 'step3model-b')
('step1model-c', 'step2model-a', 'step3model-a')
('step1model-c', 'step2model-a', 'step3model-b')

HTH
Kent