Not able to store data to dictionary because of memory limitation

[Tim Chase]

On 07/06/2011 02:49 AM, Rama Rao Polneni wrote:
> After storing 1.99 GB data in to the dictionary, python stopped to
> store the remaining data in to dictionary.
>
> Is there any alternate solution to resolve this issue. Like splitting
> the dictionaries or writing the data to hard disk instead of writing
> to memory.

Without details on the specifics of what you're storing in your 
dictionary, you might investigate the anydbm module which would 
allow you to have a disk-backed dictionary, but it really only 
works for string->string mappings.  You can use shelve/pickle to 
marshal other data-types into strings for use in such a mapping, 
but there are some odd edge-cases to be aware of (updating an 
instance of a class doesn't change the pickled object, so you 
have to intentionally re-store it after you change it; I've hit 
unexpected scoping issues with class-names; etc).  But for the 
general case, it might do exactly what you need to remove the 2GB 
cap.

-tkc