Pickling Robustly

[jepler epler]

On 24 Apr 2000 14:50:53 -0500, Glyph Lefkowitz
 <glyph at twistedmatrix.com> wrote:
>It would still be nice if there were an option to register an error
>handler to just ignore attributes which couldn't be pickled, rather
>than failing.  At this point though, I think I'd need to rewrite the
>module to get that.

This tickled another memory of mine, back when I was fiddling with
"POO", a Python-based multi-user game.  I discovered that anybody with
programming ability could add a non-picklable type to a POO object and
prevent the database from being dumped (i.e., saved) ... At the time, I
modified the pickle.py module to store a None when a non-picklable
object was encountered.  I seem to recall that Guido came up with a
superior solution, but unfortunately I can't seem to find it on deja.com
(I'm pretty sure the discussion was on the newsgroup)

Ah, here it is.  If you are using the pickle.py module (as opposed to
the cpickler), you can subclass pickle.Pickler and override the 'save'
method.  In poo, the following is done:

class PooPickler(pickle.Pickler):

        """CLASS PooPickler:
        This class extends the normal Pickler object such that datatypes
        which can't by pickled are set to None, rather than crashing
        the server.
        """

        # RADICALLY CHANGED on 4/20/98 to work with Python 1.5,
        # as per recommendation by our fearless leader (Guido)
        def save(self, object):
                try:
                        pickle.Pickler.save(self, object)
                except pickle.PicklingError:
                        pickle.Pickler.save(self, None)

This doesn't seem to be documented in the manpage for the pickle
module, but the documentation makes it clear that the cPickle.Pickler is
not a class and cannot have its save method overridden in the same way.

Rather than checking for a PicklingError, you probably want to use
type(object) is BlahType  or  isinstance(object, klass) .. but you can
decide that for yourself.

Good luck! I hope my answer is closer to what you're looking for this
time around.

Jeff