increasing the page size of a dbm store?
Barry
barry at barrys-emacs.org
Wed Nov 27 16:13:28 EST 2019
Maybe port to SQLite? I would not choose dbm these days.
Barry
> On 27 Nov 2019, at 01:48, Tim Chase <python.list at tim.thechases.com> wrote:
>
> Working with the dbm module (using it as a cache), I've gotten the
> following error at least twice now:
>
> HASH: Out of overflow pages. Increase page size
> Traceback (most recent call last):
> [snip]
> File ".py", line 83, in get_data
> db[key] = data
> _dbm.error: cannot add item to database
>
> I've read over the py3 docs on dbm
>
> https://docs.python.org/3/library/dbm.html
>
> but don't see anything about either "page" or "size" contained
> therein.
>
> There's nothing particularly complex as far as I can tell. Nothing
> more than a straightforward
>
> import dbm
> with dbm.open("cache", "c") as db:
> for thing in source:
> key = extract_key_as_bytes(thing)
> if key in db:
> data = db[key]
> else:
> data = long_process(thing)
> db[key] = data
>
> The keys can get a bit large (think roughly book-title length), but
> not huge. I have 11k records so it seems like it shouldn't be
> overwhelming, but this is the second batch where I've had to nuke the
> cache and start afresh. Fortunately I've tooled the code so it can
> work incrementally and no more than a hundred or so requests have to
> be re-performed.
>
> How does one increas the page-size in a dbm mapping? Or are there
> limits that I should be aware of?
>
> Thanks,
>
> -tkc
>
> PS: FWIW, this is Python 3.6 on FreeBSD in case that exposes any
> germane implementation details.
>
>
>
>
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
More information about the Python-list
mailing list