Writing huge Sets() to disk

Robert Brewer fumanchu at amor.org
Mon Jan 10 17:32:28 CET 2005

Martin MOKREJŠ wrote:
>   I have sets.Set() objects having up to 20E20 items,
> each is composed of up to 20 characters. Keeping
> them in memory on !GB machine put's me quickly into swap.
> I don't want to use dictionary approach, as I don't see a sense
> to store None as a value. The items in a set are unique.
>   How can I write them efficiently to disk?

got shelve*?

Robert Brewer
Amor Ministries
fumanchu at amor.org

* the "shelve" module in the standard library, that is.

More information about the Python-list mailing list