[Python-Dev] Can I make marshal.dumps() slower but stabler?

INADA Naoki songofacandy at gmail.com
Thu Jul 12 18:33:01 EDT 2018


On Fri, Jul 13, 2018 at 5:03 AM André Malo <nd at perlig.de> wrote:
>
> * INADA Naoki wrote:
>
> > Is there any real application which marshal.dumps() performance is
> > critical?
>
> I'm using it for spooling big chunks of data on disk, exactly for the reason
> that it's faster than pickle.
>
> Cheers,

Does your data contains repetition of same object (not same value)?

If yes, this change will affects you.
If no, you can use older version which doesn't have overhead of
checking object identity.

>>> x = [0]*100
>>> y = [0]*100
>>> data = [x,y,x]
>>> import marshal
>>> len(marshal.dumps(data))  # x is marshaled once
1020
>>> d[0] is d[2]
True
>>> d[0] is d[1]
False
>>> import json
>>> len(json.dumps(data))  # x is marshaled twice
906
>>> d = marshal.loads(marshal.dumps(data, 2))  # x is marshaled twice
>>> len(d)
1520
>>> d[0] is d[2]
False

-- 
INADA Naoki  <songofacandy at gmail.com>


More information about the Python-Dev mailing list