[issue5483] [PATCH]Add FastDbfilenameShelf: shelf nerver sync cache even when writeback=True

[Robert Lehmann]

Robert Lehmann <lehmannro at gmail.com> added the comment:

If I understand you correctly, your proposal is the following: use
Shelf.cache to cache *all* objects instead of only keeping live
references. Your patch retains the cache forever instead of purging it
on sync. (All these changes only apply with writeback=True, which you
enabled by default; nothing changes with writeback=False.)
This speeds up *repeated* reads/writes as they are kept in-memory
instead of querying the-- probably slow --database every time.

I do not think this is a feasible solution for two reasons:
(1) writeback was never intended to do such thing. It was introduced as
a solution to "make shelve less surprising." If you remove its
sync-on-close characteristics, shelve is as surprising as before. See
issue553171.
(2) If you intend to implement caching on *any* database I'd suggest not
using shelve for that matter. Using it for serialization is all okay but
I'd rather add an additional layer of indirection to implement caching
strategies if you want to do that The Right Way. (Shelf.cache is really
only a map of objects that were touched during runtime.)

I'm -1 on this patch but generally +1 on a generic caching wrapper.

The error you describe later on appears because Python is already
tearing down when gc'ing the Shelf (calling __del__ -> close -> sync).
With writeback=True this currently tries to pickle the cache again which
emits the error you observed. This should be handled in a separate issue.

----------
nosy: +lehmannro

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue5483>
_______________________________________