bad bsddb key/values on large data

Barry Schiffman schiffo2000 at
Tue Aug 31 14:27:04 CEST 2004

--- Skip Montanaro <skip at> wrote:

> From: Skip Montanaro <skip at>
> Date: Mon, 30 Aug 2004 21:45:28 -0500
> To: Barry Schiffman <schiffo2000 at>
> CC: python-list at
> Subject: Re: bad bsddb key/values on large data
>     Barry> The largest one, about 200 megs, is
> written, and can be read, and
>     Barry> the strings are correctly formatted --
> but the values attached to
>     Barry> the keys are wrong ...
>     ...
>     Barry> Can anyone tell me if there is such a
> Python problem or a bsddb
>     Barry> problem?
> You didn't say what versions of Berkeley DB or
> Python you are using.  If you
> are using Berkeley DB 1.85, problems are to be
> expected.  Upgrade to a later
> version.
> Skip

My apologies, but I wrote this too soon. 

The chief suspect is an interaction between one of the
hash keys and my code. The entire table is shifted
ahead on the very first key, so the value for key_1 is
saved under key_2, and then so and so forth. The shift
stays constant at one step (so key_n contains the
record for key_n-1). This only happens with one of the
three files. In fact, the code is supposed to reject
many of the records in the input, and does so without
messing up the key/value pairs -- except for this one.

However, I won't be able to check this out until

Anyway, I think the versions are OK (though I'm not
sure if the Sleepcat versions are the same as the
Berkeley versions). I tried this twice under these

Python 2.2.2, Sleepycat 3.3.11 on a PC with RH 7.3
Python 2.3.2, Sleepycat 3.2.9 on a PPC with YDL 2.3

Thanks much.


Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage! 

More information about the Python-list mailing list