[Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

[Guido van Rossum]

On Thu, Nov 4, 2010 at 1:25 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
> On Nov 4, 2010, at 12:49 PM, Guido van Rossum wrote:
>
> What's the attack you're thinking of on marshal? It never executes any
> code while unmarshalling (although it can unmarshal code objects --
> but the receiving program has to do something additionally to execute
> those).
>
> These issues may have been fixed now, but a long time ago I recall seeing
> some nasty segfaults which looked exploitable when feeding marshal malformed
> data.  If they still exist, running a fuzzer on some pyc files should reveal
> them pretty quickly.
>
> When I ran across them I didn't think much of them, and probably did not
> even report the bug, since marshal is mostly used to load code anyway, which
> is implicitly trusted.

I'm not sure that all these were fixed but it would be a finite (and
probably small) amount of work to get it fixed -- unlike fixing
pickling, which is impossible (unless you implemented some kind of
sandboxing solution :-).

A good use for pickling is when it's optional. Example: putting
pickles in memcache. The source of the pickles is (presumably)
trusted, so the only remaining problem is occasional version skew. If
the unpickling fails it can just be treated as a cache miss. (Tricky:
when unpickling succeeds but returns a broken object. "Nobody's
perfect." :-)

-- 
--Guido van Rossum (python.org/~guido)