Thanks for the proposal Mark!
I wanted to make some comments regarding converting AST nodes to PyObjects internally. I see some challenges here:
* Not using an arena allocator for the nodes can introduce more challenges than simplifications. The first is that deleting a deep tree currently is just freeing the arena block, while if the nodes were PyObjects it will involve recursive destruction. That could potentially segfault so we would need to use some custom trashcan mechanism of special deleters. All of this will certainly not simplify the code (at least the parser code) and will impact performance (although just in the parser/compiler phase).
* We would need to (potentially) reimplement the AST sequences into proper owning-containers. That will involve changing a considerable amount of code and some slowdown due to having to use C-API calls.
* The proposal seems to imply that the AST will be a fully public and stable API. This has some danger as any internal optimization regarding AST can be braking changes to macro users. This will raise any problems that now linter and static analyst tools could have to all users (of macros), making of even more difficult to change it.
Regards from sunny London,
Pablo Galindo