[Python-Dev] Pickle alternative in stdlib (Was: On breaking modules into packages)

[anatoly techtonik]

On Thu, Nov 4, 2010 at 3:38 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Thu, Nov 4, 2010 at 4:28 PM, anatoly techtonik <techtonik at gmail.com> wrote:
>> On Wed, Nov 3, 2010 at 9:08 PM, Glyph Lefkowitz <glyph at twistedmatrix.com> wrote:
>>>
>>> This is the strongest reason why I recommend to everyone I know that they
>>> not use pickle for storage they'd like to keep working after upgrades [not
>>> just of stdlib, but other 3rd party software or their own software]. :)
>>>
>>> +1.
>>> Twisted actually tried to preserve pickle compatibility in the bad old days,
>>> but it was impossible.  Pickles should never really be saved to disk unless
>>> they contain nothing but lists, ints, strings, and dicts.
>>
>> But what is alternative in stdlib?
>> Don't you think that Python doesn't provide any?
>
> Python 3.2a3+ (py3k:85817, Oct 24 2010, 19:25:28)
> [GCC 4.4.3] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import json
>>>> dir(json)
> ['JSONDecoder', 'JSONEncoder', '__all__', '__author__',
> '__builtins__', '__cached__', '__doc__', '__file__', '__name__',
> '__package__', '__path__', '__version__', '_default_decoder',
> '_default_encoder', 'decoder', 'dump', 'dumps', 'encoder', 'load',
> 'loads', 'scanner']
>
> pickle gets overspecific in many ways, and hence (despite our best
> efforts, and those of third parties) may break when changing Python
> versions. Serialising to something more language natural (be it JSON,
> YAML, XML or one of the multitude of other state encoding formats out
> there) is far more likely to be future proof.
>
> As a tool for communicating between different instances of the *same*
> version of Python though, pickle is fine.

pickle is insecure, marshal too. What about JSON? IIUC you need a
definition of a class to be able to unserialize it in all cases. I
wonder how is this definition validated, i.e. what to watch for when
modifying classes that can be serialized.
--
anatoly t.