[Python-Dev] RFC: PEP 445: Add new APIs to customize Python memory allocators

Victor Stinner victor.stinner at gmail.com
Thu Jun 20 07:37:42 CEST 2013

Le jeudi 20 juin 2013, Nick Coghlan a écrit :
> > Is PyMemMappingAllocator complete enough for your usage at CCP Games?
> Can we go back to calling this the "Arena" allocator? Or at least
> "Mapped"? When I see "Mapping" in the context of Python I think of the
> container API, not a memory allocation API.
This function is written to be able to use mmap() and VirtualAlloc(). There
is no Python function to use directly this allocator yet, but I chose
"memory mapping" name because it is very different than the heap and it may
be useful for other functions than pymalloc.

If I change the name, it would be called PyObject_SetArenaAllocator() with
a PyObjectArenaAllocator structure. I'm not sure that PyMemMappingAllocator
API is future-proof, so I'm fine to call it "arena" again.

> > I hope that the PEP 445 is flexible enough to allow you to decide
> > which functions are hooked and replaced, and which functions will be
> > leaved unchanged. That's why I'm not in favor of the "Make
> > PyMem_Malloc() reuse PyMem_RawMalloc() by default" alternative.
> It's also why I'm in favour of the "domain" API rather than separate
> functions.
> 1. In the initial iteration, just have the three basic domains (raw,
> interpreter, objects). Replacing allocators for third party libraries is
> the responsibility of embedding applications.
> 2. In a later iteration, add "PyMem_AddDomain" and "PyMem_GetDomains" APIs
> so that extension modules can register new domains for wrapped libraries.
> Replacing allocators is still the responsibility of embedding applications,
> but there's a consistent API to do it.
> (Alternatively, we could do both now)
How would you use an allocator of a new domain? PyMemBlockAllocator
structure is not convinient, and if Py_GetAllocator() only once, you may
loose a hook installed later.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130620/b3b40587/attachment-0001.html>

More information about the Python-Dev mailing list