[New-bugs-announce] [issue15535] Fix pickling of named tuples in 2.7.3

[Thomas Miedema]

New submission from Thomas Miedema:

Pickling a namedtuple Point(x=10, y=20, z=30) in Python 2.7.2 with protocol level 0 would result in something like the following output:

  ccopy_reg
  _reconstructor
  p0
  (c__main__
  Point
  p1
  c__builtin__
  tuple
  p2
  (I10
  I20
  I30
  tp3
  tp4
  Rp5
  .

In Python 2.7.3, the same namedtuple dumps to:

  ccopy_reg
  _reconstructor
  p0
  (c__main__
  Point
  p1
  c__builtin__
  tuple
  p2
  (I10
  I20
  I30
  tp3
  tp4
  Rp5
  ccollections
  OrderedDict
  p6
  ((lp7
  (lp8
  S'x'
  p9
  aI10
  aa(lp10
  S'y'
  p11
  aI20
  aa(lp12
  S'z'
  p13
  aI30
  aatp14
  Rp15
  b.

Note the OrderedDictionary at the end. All data, the field names and the values, are duplicated, which can result in very large pickled files when using nested namedtuples.

Loading both dumps with CPython 2.7.3 works. This is why this bug was not noticed any earlier. Loading the second dump with CPython or pypy 2.7.2 does not work however. CPython 2.7.3 broke forward compatibility.

Attached is a patch with a fix. The patch makes pickled namedtuples forward compatibile with 2.7.2. This patch does not break backward compability with 2.7.3, since the extra OrderedDict data contained the same information as the tuple. 

Introduced:
http://hg.python.org/cpython/diff/26d5f022eb1a/Lib/collections.py

Also relevant:
http://bugs.python.org/issue3065

----------
components: Library (Lib)
files: namedtuple_pickle_fix.patch
keywords: patch
messages: 167215
nosy: rhettinger, thomie
priority: normal
severity: normal
status: open
title: Fix pickling of named tuples in 2.7.3
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file26662/namedtuple_pickle_fix.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15535>
_______________________________________