Best way to add a "position" value to each item in a list

[Carl Banks]

On Jul 9, 1:16 pm, Sean <sberr... at gmail.com> wrote:
> I have a huge list, 10,000,000+ items.  Each item is a dictionary with
> fields used to sort the list.  When I have completed sorting I want to
> grab a page of items, say 1,000 of them which I do easily by using
> list_data[x:x+1000]
>
> Now I want to add an additional key/value pair to each dictionary in
> the list, incrementing them by 1 each time.  So, if I grabbed page 2
> of the list I would get:
>
> [{'a':'a', 'b':'b', 'position':1001}, {'c':'c', 'd':'d', 'position':
> 1002}, ...]
>
> Any way to do that with list comprehension?  Any other good way to do
> it besides iterating over the list?


Not really, but if you want to avoid the delay in setting the
dictionary elements (I'm guessing that is why you don't want to
iterate over the list--and parenthetically it's not iterating over the
list but adding dictionary items that is driving the time, and that
cost is unavoidable), you should consider adding the position elements
on demand.  That is, instead of running a big loop once to add
position to all ten million elements, just run the loop on individual
pages.

def get_page(start,end):
    page = list_data[start:end]
    for i,item in enumerate(page):
        page['position'] = start+i
    return page


That might not work depending on what you are doing but you should
consider it.


Carl Banks