Re: [Python-ideas] SQL-like way to manipulate Python data structures

May 26, 2007

      --- Arnaud Delobelle <arno@marooned.org.uk> wrote:
...
On 26 May 2007, at 17:06, Steve Howell wrote:
...
I find myself often writing somewhat tedious code
in
Python that is just slicing/aggregating fairly
simple
data structures like a list of dictionaries.
Here is something that I'm trying to express,
roughly:
charges =
       sum float(charge) from events
       on setup, install
       group by name
I don't really understand that :)
Let me try again:

select
    name,
    bucket,
    sum(float(charge))
from events
where bucket in ('install', 'setup')
group by name, bucket

The idea here is that events is a list of Python
dictionaries keyed by fields "name", "bucket", and
"charge", and I only care about install/setup charges.

Here is a rough Python statement of what I'm trying to
do:

charges = {}
for name, n_evts in groupby(events,
itemgetter('name')):
    print name
    print dict([(bucket, sum([float(e['charge']) for e
in b_evts])) for
            bucket, b_evts in groupby(n_evts,
itemgetter('bucket'))
            if bucket in ('setup', 'install')])
...
...
Here is the Python code that I wrote:
def groupDictBy(lst, keyField):
        dct = {}
        for item in lst:
            keyValue = item[keyField]
            if keyValue not in dct:
                dct[keyValue] = []
            dct[keyValue].append(item)
        return dct
isn't that itertools.groupby?
The problem with itertools.groupby is that it assumes
sorted lists.  So the code below does not behave how
you might expect.  It reports Joe's charges twice, and
it loses Steve's $50 install charge.

from itertools import groupby
from operator import itemgetter

events = [
        {'name': 'Joe',   'bucket': 'install',
'charge': 100},
        {'name': 'Joe',   'bucket': 'setup',  
'charge': 20},
        {'name': 'Steve', 'bucket': 'install',
'charge': 50},
        {'name': 'Steve', 'bucket': 'setup',  
'charge': 30},
        {'name': 'Steve', 'bucket': 'install',
'charge': 1000},
        {'name': 'Steve', 'bucket': 'upgrade',
'charge': 440},
        {'name': 'Joe',   'bucket': 'setup',  
'charge': 40},
        ]

charges = {}
for name, n_evts in groupby(events,
itemgetter('name')):
    print name
    print dict([(bucket, sum([float(e['charge']) for e
in b_evts])) for
            bucket, b_evts in groupby(n_evts,
itemgetter('bucket'))
            if bucket in ('setup', 'install')])

____________________________________________________________________________________Pinpoint customers who are looking for what you sell. 
http://searchmarketing.yahoo.com/