On Feb 7, 2020, at 09:49, Daniel Spitz <spitz.dan.l@gmail.com> wrote:
But, it uses pickle under the hood, because pickle is what knows how to traverse object graphs with cycles, and pickle is what knows how to use the extensible __reduce__ machinery. And this makes the entire demo silly and really slow. (And, that was kind of the point, to do something interesting using esoteric, inappropriate corners of python.)
If Andrew's proposal happened, pickle would be uncoupled from that core machinery, and my idea, as well as other interesting ideas, could be implemented in a non-silly way.
Not to argue against my own proposal here, but…i was only suggesting exposing the reduction machinery, and I think you also need the graph-walking machinery exposed. Or would my existing proposal on its own actually be enough for you? That being said, if you have a use for the object graph walk, that shouldn’t be too hard to write. It’s just that there are nontrivial design issues to answer. Do we need to expose the graph itself in some form, or just the memoized walk iterator? Is an Iterator the right way to expose it, as opposed to something like the Smalltalk-ish walker classes in the AST module? Should it iterate trivial objects (the core builtin types that pickle knows it doesn’t need to further reduce—but you can get their reduction anyway if you want)? Do we need to expose a function to let user code check whether an object is trivial in this sense? Should it Iterate the objects themselves, or some kind of node object? Does it need to (optionally?) yield anything for references to already-visited nodes—and, if so, what does that look like? Should pickle and deepcopy be rewritten to use the public walker, or should they continue to inline the walk throughout their code and the walker is just a parallel implementation of the same logic—and, if the latter, do we need to guarantee that it always implements identical logic, or just “good“ logic? I took only the briefest look at your code (I’m on my phone); it’s possible that some of these already have obvious “it has to be this answer or I can’t write my transformer” answers. It’s also possible that the some of these questions have answers that aren’t obvious from first principles but would become obvious as soon as we start implementing it. And this might be a fun weekend project, so maybe it’s worth just diving in to see what we run into, and then just ask -ideas about the remaining open questions. If you don’t want to try it, there’s a good chance I will, but no promises as to when.