[Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 - serializers.py:1.3storage.py:1.8

[Tim Peters]

[Shane Hathaway]
> Modified Files:
> 	serializers.py storage.py
> Log Message:
> Fixed Python 2.3 compatibility.
>
> - If you append to a list while it is being pickled, Python 2.3 will
>   pick up the new items.  Previous versions of Python did not.

How do you grow a list while it's being pickled?  Must be another thread
doing that, and if so that's mighty dubious.  If you're pickling to a
genuine file object, then cPickle releases the GIL around its fwrite()
calls, so that's one way.  But in the context of the patch:

+        p.persistent_id = lambda ob: None  # Stop recording references
         p.dump(unmanaged)
         s = outfile.getvalue()
         event.addUnmanagedPersistentObjects(unmanaged)

the use of .getvalue() suggests the pickle target isn't a real file.  Then
cPickle never releases the GIL (cStringIO doesn't either), and no other
thread should dare muck with a Python object (like the Python list getting
pickled) without holding the GIL.

However this ends up happening, if you instead shrink a list while it's
being pickled, Pythons earlier than 2.3 can end up trying to pickle random
recycled memory.  Their cPickles captured the list length once at the start
of pickling the list, and that coding optimization is appropriate only if
the GIL is never released.  2.3 uses the iteration protocol instead, so
"sees" dynamic changes in the list size.

Copying Python-Dev because this may reveal a new mountain of ways to crash
the interpreter:  mutate objects in devious ways while they're getting
cPickled.