Is there any library for indexing binary data?
Albert van der Horst
albert at spenarnc.xs4all.nl
Sun Apr 4 13:28:15 CEST 2010
In article <mailman.1173.1269496428.23598.python-list at python.org>,
=?GB2312?B?zPC5zw==?= <littlesweetmelon at gmail.com> wrote:
>Well, Database is not proper because 1. the table is very big (~10^9
>rows) 2. we should support very fast *simple* query that is to get
>value corresponding to single key (~10^7 queries / second).
>Currently, I have implemented a specific algorithm to deal with my
>problem. However, I want to employ some library to simplify codings,
>otherwise I have to write my own code for each big table. It is
>possible that, after using indexing library, the program cannot run as
>fast as homemade code. But if it can greatly simplify my job and can
>provide satisfied speed (eg 10^5~10^6 queries / second), the indexing
>library is still a good choice for me.
At first sight this looks to me like B-trees would be an ideal
solution. The first levels of the tree are in memory, so with some
luck you have only one or two disk accesses per search.
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
More information about the Python-list