[Python-ideas] Adding a safe alternative to pickle in the standard library

[Devin Jeanpierre]

On Fri, Feb 22, 2013 at 7:29 AM, Stephen J. Turnbull <stephen at xemacs.org> wrote:
>  > That does not mean that we should not write or use safer alternatives.
>  > We have written and do use safer alternatives, like the json module.
>
> Then why do we need a "safe alternative to pickle" when json is
> already in the standard library?

json can't handle cycles or new types -- at least, not without a
rather significant amount of preprocessing and postprocessing that is
little less work than writing your own serialization library from
scratch.

I do believe that support for these things is important. People do
want to store frozensets of tuples, and objects which reference
themselves, and so on. The easy way to do this is with pickle, and
that encourages people to use pickle even where it is a bad idea. The
easier it is to do something safer, the more likely it is people will
do something safer. I've seen people use pickle even when advised that
it was a security risk, because they didn't want to go through the
effort of using the json module. To me that signals that something is
wrong.

> Well, we've already gone as far as json, which is pretty powerful (but
> still subject to attacks using "relatively secure" json to transport
> "insecure" data!)

Of course a serialization library can't protect against
eval(deserialize(foo)) running arbitrary code. That doesn't mean we
shouldn't bother with security.

> Why do we need an alternative *between* pickle and
> json?  Maybe we should advocate that users think seriously about
> securing channels, and validating the pickles before doing anything
> with them, if they think they need more features than json offers?

Signed pickles and secured channels and so on don't solve the problem
of trying to get data from an an untrusted user. Moreover I think it
isn't an uncommon opinion that, even in that case, it'd be better to
use json or some other nominally secure library instead of pickle,
since it increases the number of mistakes necessary to compromise your
security.

-- Devin