[Python-ideas] Adding a safe alternative to pickle in the standard library

[Stephen J. Turnbull]

Steve Dower writes:

 > I was only aiming to provide perspective, rather than a proposal.

Sure, but I didn't think there was a need for more "general"
perspective.  Pickle is a well-established protocol with certain risks
and certain benefits, of which the python-dev community is basically
well-aware.  This is coming up *now* because recent events (Ruby/YAML
has been mentioned) are causing some people to reevaluate the risks.
This is a "quantitative" issue, and needs concrete proposals.

 > To be really explicit, I would make load/loads only work with
 > built-in types. For compatibility when reading earlier protocols we
 > could add a type representing a class instance and its members that
 > doesn't actually construct anything. (Maybe we override __getattr__
 > to make it lazily construct the instance when the program actually
 > wants to use it?)

I am not a security expert, but it seems to me that's going in the
wrong direction.  Unpickler would *still* run constructor code
automatically under some circumstances -- but those circumstances
become murkier.

 > For convenience, I'd add a parameter to Unpickler to let the user
 > provide a set of types that are allowed to be constructed (or a
 > mapping from names to callables as used in find_class()).

And this is secure, why?  There's no way to decorate the allowed types
to add nasty stuff to the pickled class definitions (including
built-in types), right?  There are no bugs that allow a back door,
right?  Is the API sufficiently well-designed that users will easily
figure out how to do what they need, and *only* what they need, and
therefore won't be tempted to simply turn on permission to do
*everything*?  And they won't give up, and write their own?

Isn't it better just to give users the advice to use JSON where it
will do?  Perhaps the difference in APIs will give them pause to think
again if they're starting to think about unpickling classes?

Granted, I don't have answers to those questions (except for myself!)
But I think some thought should be given to them before trying to
create a restricted pickle protocol and make it default.  Restricted
modes/protocols/sublanguages are hard to get right.