On Tue, Jul 14, 2020, at 21:24, Chris Angelico wrote:
I actively oppose it because it isn't possible. Anything that is safe will not have all of pickle's functionality. A nerfed version of pickle that can only unpickle a tiny handful of core data types is no better than other options that already exist. The entire point of pickling arbitrary objects is that you can unpickle arbitrary objects.
I don't understand why no-one's engaging with what I actually suggested. I was not asking for a magically safe or arbitrarily restricted pickle function.
I was asking for the current Unpickler class, which currently has a whitelist hook for loading globals, to be modified to also have a whitelist hook so that an application can provide a function that looks at a callable and its arguments that the pickle proposes to call, and can choose to either evaluate it, raise an error, or return a substitute value.
That's inherently unsafe if there is any possibility that the pickle file came from an untrusted user, and I do indeed oppose plans to try to make pickle what it isn't.
We already have one whitelist hook, why not another?
The idea that the pickle format is "inherently unsafe" and cannot be made safe is magical thinking. What I am asking for is the ability for application code subclassing Unpickler to control how certain opcodes are evaluated by overriding methods... something that *already exists*, just not for the right ones needed to be adequately expressive.
To the person who ran a fuzzer against it and found inputs that can cause segfaults or MemoryError, those are no more inherent to pickle than an equivalent bug in the JSON parser would be to JSON. They are bugs which can and should be fixed.