Would it complicate the parser much to allocate memory using a different
allocator? It's just a function call, presumably.
Is not the allocator per-se, is the fact that with the arena we can free *all the memory* in one single call, instead of having to implement custom free functions for the nodes or triggering a recursive chain of deallocations. The parser itself owns all the memory block (by owning the arena) so we don't need to add specific ownership models to the nodes, which makes the inner workings much easier to organise, including temporary allocations.
As for performance, we don't know how much difference it will make until
Deallocation doesn't have to be recursive.
But if we model the nodes as Python objects with owership via attributes, it will be recursive because it's a tree. You would need specific allocators to avoid it.
On Wed, 16 Sep 2020 at 12:48, Mark Shannon firstname.lastname@example.org wrote:
On 16/09/2020 12:22 pm, Pablo Galindo Salgado wrote:
> Don't we need to do all of this in the _ast module, already? > We already have an AST composed of Python objects
We have AST of Python objects, but the python versions are not used internally, especially in the parser, where they are created. The parser and the compiler currently use exclusively the C versions of the nodes. Changing the
to use Python versions and especially not to use the arena allocator can complicate the parser substantially, and destroying the resulting tree can be expensive (as destroying a deep tree composed of Python objects currently is).
Would it complicate the parser much to allocate memory using a different allocator? It's just a function call, presumably.
As for performance, we don't know how much difference it will make until we try. Deallocation doesn't have to be recursive.