itertools: problem with nested groupby, list()

[Nico Schlömer]

Hi,

I ran into a bit of an unexpected issue here with itertools, and I
need to say that I discovered itertools only recently, so maybe my way
of approaching the problem is "not what I want to do".

Anyway, the problem is the following:
I have a list of dictionaries, something like

[ { "a": 1, "b": 1, "c": 3 },
  { "a": 1, "b": 1, "c": 4 },
  ...
]

and I'd like to iterate through all items with, e.g., "a":1. What I do
is sort and then groupby,

my_list.sort( key=operator.itemgetter('a') )
my_list_grouped = itertools.groupby( my_list, operator.itemgetter('a') )

and then just very simply iterate over my_list_grouped,

for my_item in my_list_grouped:
    # do something with my_item[0], my_item[1]

Now, inside this loop I'd like to again iterate over all items with
the same 'b'-value -- no problem, just do the above inside the loop:

for my_item in my_list_grouped:
        # group by keyword "b"
        my_list2 = list( my_item[1] )
        my_list2.sort( key=operator.itemgetter('b') )
        my_list_grouped = itertools.groupby( my_list2,
operator.itemgetter('b') )
        for e in my_list_grouped:
            # do something with e[0], e[1]

That seems to work all right.

Now, the problem occurs when this all is wrapped into an outer loop, such as

for k in [ 'first pass', 'second pass' ]:
    for my_item in my_list_grouped:
    # bla, the above

To be able to iterate more than once through my_list_grouped, I have
to convert it into a list first, so outside all loops, I go like

my_list.sort( key=operator.itemgetter('a') )
my_list_grouped = itertools.groupby( my_list, operator.itemgetter('a') )
my_list_grouped = list( my_list_grouped )

This, however, makes it impossible to do the inner sort and
groupby-operation; you just get the very first element, and that's it.

An example file is attached.

Hints, anyone?

Cheers,
Nico
-------------- next part --------------
A non-text attachment was scrubbed...
Name: iterator-test.py
Type: text/x-python
Size: 1146 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-list/attachments/20100504/2083f98c/attachment.py>