[Python-Dev] Pickler/Unpickler API clarification

[Guido van Rossum]

On Thu, Mar 5, 2009 at 1:36 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
>> This was a bad idea (*), and I'd be happy to ban it -- but we'd
>> probably have to bump the pickle protocol version in order to maintain
>> backwards compatibility.
>
> If you're talking about multiple calls to dump() on the same
> pickler, it might be a bad idea for a network connection, but
> I don't see anything wrong with using it on a file, and I find
> it useful to do so sometimes. Banning it would be excessive, IMO.

I don't think I was thinking of that when I first designed pickle but
the use case makes some sense. I still wish we could ban it or somehow
make it *not* the default behavior; the bug in the App Engine bug I
referenced before was introduced by an experienced developer who
wasn't aware of this behavior and was simply trying to avoid
unnecessarily creating a new pickler for each call.

>> The exposition is unintentional but for historic reasons we can't just
>> remove it. :-(
>
> A compromise might be to provide a memo attribute that returns
> a wrapper around the underlying cache -- maybe with only a
> clear() method if that's all you want to support.

Then it'd be better to have a method clear_memo() on pickle objects.

Perhaps we should do the opposite, and have a separate API for reuse
*without* clearing the memo? <pickler>.dump_reusing_memo(<value>) and
<unpickler>.load_reusing_memo().

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)