[Patches] marshalling recursive objects

[Guido van Rossum]

> I saw recently in c.l.py a discussion about recursive objects. It was
> noted that recursive comparisons were fixed, but marshalling recursive
> objects still core dumps.
> 
> >>> l = []
> >>> l.append(l)
> >>> marshal.dumps(l)
> Paf!
> 
> The following patch aims at fixing this. The approach is the same as the
> one used in pickle (I believe). Container objects are kept in an internal
> list. When serializing, on recursion branches, the list index of the container
> is stored in the byte stream, instead of its contents. This index should
> be the same when deserializing, because of the strict object ordering.
> Recursions are thus restored, and we have the invariant:
> 
>    o == marshal.loads(marshal.dumps(o))
> 
> even when o involves cyclic references.

Vladimir,

Thanks for this patch.  However, I'm worried that it does too much!
I'd be happy if marshalling a recursive data structure just raised an
exception.  I'm happy enough to see a solution that's better, but
I'm not sure I like it to write different marshalling strings for data
structures that aren't recursive.

Here's an example:

>>> a = []
>>> b = [a,a,a]
>>> marshal.dumps(b)
'[\003\000\000\000[\000\000\000\000o\001\000\000\000o\001\000\000\000'
>>> 

Before your patch, this produced:

'[\003\000\000\000[\000\000\000\000[\000\000\000\000[\000\000\000\000'

Why do I care?  Marshals may be used for data interchange between
applications running on different machines and using different Python
versions.

I realize this is harder, and I understand that in some sense your
solution is "better" -- but for the semantics that you are creating,
we already have a better solution: pickle.  The marshal module is
about values, not objects.

I'm not sure how to fix it -- if the easiest solution is to raise an
exception for recursive objects, that's fine.  (It's a heck of a lot
better than Memory fault!)

--Guido van Rossum (home page: http://www.python.org/~guido/)