[Patches] marshalling recursive objects
Guido van Rossum
guido@python.org
Fri, 12 May 2000 13:48:56 -0400
> I saw recently in c.l.py a discussion about recursive objects. It was
> noted that recursive comparisons were fixed, but marshalling recursive
> objects still core dumps.
>
> >>> l = []
> >>> l.append(l)
> >>> marshal.dumps(l)
> Paf!
>
> The following patch aims at fixing this. The approach is the same as the
> one used in pickle (I believe). Container objects are kept in an internal
> list. When serializing, on recursion branches, the list index of the container
> is stored in the byte stream, instead of its contents. This index should
> be the same when deserializing, because of the strict object ordering.
> Recursions are thus restored, and we have the invariant:
>
> o == marshal.loads(marshal.dumps(o))
>
> even when o involves cyclic references.
Vladimir,
Thanks for this patch. However, I'm worried that it does too much!
I'd be happy if marshalling a recursive data structure just raised an
exception. I'm happy enough to see a solution that's better, but
I'm not sure I like it to write different marshalling strings for data
structures that aren't recursive.
Here's an example:
>>> a = []
>>> b = [a,a,a]
>>> marshal.dumps(b)
'[\003\000\000\000[\000\000\000\000o\001\000\000\000o\001\000\000\000'
>>>
Before your patch, this produced:
'[\003\000\000\000[\000\000\000\000[\000\000\000\000[\000\000\000\000'
Why do I care? Marshals may be used for data interchange between
applications running on different machines and using different Python
versions.
I realize this is harder, and I understand that in some sense your
solution is "better" -- but for the semantics that you are creating,
we already have a better solution: pickle. The marshal module is
about values, not objects.
I'm not sure how to fix it -- if the easiest solution is to raise an
exception for recursive objects, that's fine. (It's a heck of a lot
better than Memory fault!)
--Guido van Rossum (home page: http://www.python.org/~guido/)