how to build a dict including a large number of data
wanzathe
wanzathe at gmail.com
Fri Jan 4 08:57:50 EST 2008
hi everyone
i'm a newbie to python :)
i have a binary file named test.dat including 9600000 records.
the record format is int a + int b + int c + int d
i want to build a dict like this: key=int a,int b values=int c,int d
i choose using bsddb and it takes about 140 seconds to build the dict.
what can i do if i want to make my program run faster?
or is there another way i can choose?
Thanks in advance.
My Code:
-----------------------------------------------------------------------------------
my_file = file('test.dat','rb')
content = my_file.read()
record_number = len(content) / 16
db = bsddb.btopen('test.dat.db','n',cachesize=500000000)
for i in range(0,record_number):
a = struct.unpack("IIII",content[i*16:i*16+16])
db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3])
db.close()
my_file.close()
More information about the Python-list
mailing list