[Christian]
Several people asked on the main list, how to pickle deeply nested structures without crashing pickle. Well, my general answer was to rewrite pickle in a non-recursive manner.
[Guido]
I guess it's my anti-Scheme attitude. I just think the problem is in the deeply nested structures. There usually is a less nested data structure that doesn't have the problem. But I'll shut up, because this rant is not productive. :-(
Ya, but it *used* to be -- in the early days, many people learned a lot about writing better programs by avoiding constructs Python penalized (nested functions, cyclic references, deep recursion, very long reference chains, massively incestuous multiple inheritance). Learning to design with flatter data structures and flatter code was highly educational, and rewarding, at least for those who played along. I suppose that's gone for good now. An irony specific to pickle is that cPickle coding was driven mostly by Zope's needs, and multi-gigabyte Zope databases live happily with its recursive design -- most data ends up in BTrees, and those hardly ever go deeper than 3 levels. I don't think it's coincidence that, needing to find a scalable container type with demanding size and speed constraints, Jim ended up with a "shallow" BTree design. The lack of need for deep C recursion was then a consequence of needing to avoid (for search speed) long paths from root to data. Oh well. The next generation will learn the hard way <wink>. looking-forward-to-death-ly y'rs - tim