Arghh! This is in fact harder than I was aware of. You *have* a setter, for its existance, although it won't set, for the readonly flag. Without criticism, you are for sure not finally happy with the solution, which sounds more like a working proof of concept than a real concept which you are happy to spread on the world. I'm better off to keep my hands off and not touch it now.
Actually, I like it fine. There really are four categories: 0) not a descriptor 1) overridable descriptor (used for methods) 2a) read-only non-overridable descriptor (used for read-only data) 2b) writable non-overridable descriptor (used for writable data) Case (0) is recognized by not having __get__ at all. Case (1) has __get__ but not __set__. Cases (2a) and (2b) have __get__ and __set__; case (2a) has a __set__ that raises an exception. There are other (older) examples of __setattr__ implementations that always raise an exception.
I patches pickle.py and cPickle.c to do essentially what Armin said: """ So I'm just saying that pickle.py in wrong in just one place:
reduce = getattr(obj, "__reduce__", None) if reduce: rv = reduce()
should be:
reduce = getattr(type(obj), "__reduce__", None) if reduce: rv = reduce(obj) """
Right. (That's what I was trying to say, too. :-)
An almost trivial change, although I also had to change copy.py, and overall I was unhappy since this extends my patch set to more than replacing python2x.dll, but I hope this will become an official patch and back-patch.
Give it to me baby. (On SF. :-)
What's the reason for wanting to make cPickle non-recursive?
Several reasons. For one, the same reason why I started arguing about deeply recursive destruction code, and implemented the initial elevator destructor, you remember. (trashcan)
Yeah. Maybe I should get out of the serious language implementation business, because I still liked it better before. It may work, but it is incredibly ugly, and also had bugs for the longest time (and those bugs were a lot harder to track down than the bug it was trying to fix). With Tim's version I can live with it -- but I just don't like this kind of complexification of the implementation, even if it works better.
Same reason. When __del__ crashes, cPickle will crash as well.
Please don't call it __del__. __del__ is a user-level finalization callback with very specific properties and problems. You were referring to tp_dealloc, which has different issues.
Now that I *can* pickle tracebacks and very deep recursions, I don't want them to crash.
Several people asked on the main list, how to pickle deeply nested structures without crashing pickle. Well, my general answer was to rewrite pickle in a non-recursive manner.
I guess it's my anti-Scheme attitude. I just think the problem is in the deeply nested structures. There usually is a less nested data structure that doesn't have the problem. But I'll shut up, because this rant is not productive. :-(
On the other hand, my implementation for tracebacks and tasklets (with large chains of frames attached) was different: In order to avoid cPickle's shortcomings of recursion, I made the tasklets produce a *list* of all related frames, instead of having them refer to each other via f_back. I did the same for tracebacks, by making the leading traceback object special, to produce a *list* of all other traceback objects in the chain.
Hey, just what I said. :-)
Armin once said, "rewrite the pickle code", which I'd happily do, but I do think, the above layout changes are not that bad, anyway. WHile frame chains and traceback chains are looking somewhat recursive, they aren't really. I think, they are lists/tuples by nature, and pickling them as that not only makes the result of __reduce__ more readable and usable, but the pickle is also a bit shorter than that of a deeply nested structure.
Well, unclear. Frame chains make sense as chains because they are reference-counted individually.
Well, after Armin's input, I dropped my special case, and instead I will submit a patch for 2.2 and 2.3, which uses your proposed way to use __reduce__ from pickle and copy. This is completely compatible and does what we want!
Wonderful! Please send me the SF issue, I don't subscribe to SF any more. (I've done my checkin in case you wondered.) --Guido van Rossum (home page: http://www.python.org/~guido/)