<div dir="ltr">fwiw i'm also supportive of adding these apis. Lets PEP away to iron out any details or document disagreements but overall I'd also like to see something a lot like these go in.<div><br></div><div>-gps</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jun 14, 2013 at 10:50 PM, Nick Coghlan <span dir="ltr"><<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 15 June 2013 11:54, Victor Stinner <<a href="mailto:victor.stinner@gmail.com">victor.stinner@gmail.com</a>> wrote:<br>
> 2013/6/15 Antoine Pitrou <<a href="mailto:solipsis@pitrou.net">solipsis@pitrou.net</a>>:<br>
>>> <a href="http://hg.python.org/cpython/rev/6661a8154eb3" target="_blank">http://hg.python.org/cpython/rev/6661a8154eb3</a><br>
>>> ...<br>
>>> Issue #3329: Add new APIs to customize memory allocators<br>
>>><br>
>>> * Add a new PyMemAllocators structure<br>
>>> * New functions:<br>
>>><br>
>>> - PyMem_RawMalloc(), PyMem_RawRealloc(), PyMem_RawFree(): GIL-free memory<br>
>>> allocator functions<br>
>>> - PyMem_GetRawAllocators(), PyMem_SetRawAllocators()<br>
>>> - PyMem_GetAllocators(), PyMem_SetAllocators()<br>
>>> - PyMem_SetupDebugHooks()<br>
>>> - _PyObject_GetArenaAllocators(), _PyObject_SetArenaAllocators()<br>
>><br>
>> My two cents, but I would prefer if this whole changeset was reverted.<br>
>> I think it adds too much complexity in the memory allocation APIs,<br>
>> for a pretty specialized benefit. IMHO, we should be able to get by with<br>
>> less allocation APIs (why the new _Raw APIs) and less hook-setting<br>
>> functions.<br>
><br>
> Ok, I reverted my commit.<br>
><br>
> I posted my initial patch 3 months ago on the bug tracker. I got some<br>
> reviews and discussed with Kristján Valur Jónsson who heavily modified<br>
> Python for his game at CCP. I started two threads on python-dev this<br>
> week (ok, only two days ago). I thaugh that the last known issues were<br>
> fixed with the addition of PyMem_SetupDebugHooks() (to avoid an<br>
> environment variable, as asked by Nick) and PyMem_RawMalloc() (have a<br>
> GIL-free allocator).<br>
><br>
> I will work on a PEP to explain all these new functions and their use cases.<br>
<br>
</div></div>I think the new APIs are mostly valid and well-justified, but agree a<br>
PEP is a good idea.<br>
<br>
Yes, it's a complex solution, but it's solving a complex problem that<br>
arises when embedding CPython inside executables that need to run on<br>
non-traditional platforms where the simple C defined<br>
malloc/realloc/free trio is inadequate.<br>
<br>
This is a complementary effort to PEP 432 - that aims to simplify<br>
embedding CPython in general, while Victor's efforts here specifically<br>
focus on situations where it is necessary to better map CPython to an<br>
underlying memory model that differs from the traditional C one. While<br>
the "single heap" model of memory enshrined in the C standard is<br>
certainly the most common model, it's far from being the only one, and<br>
these days CPython also gets used in those other environments.<br>
<br>
About the only simplification I can see is that PyMem_RawMalloc(),<br>
PyMem_RawRealloc(), PyMem_RawFree() could perhaps be handled with<br>
preprocessor macros instead of permitting runtime reconfiguration.<br>
Allowing the memory allocations for the CPython runtime to be handled<br>
separately from those for arbitrary C libraries loaded into the<br>
process seems reasonable, though.<br>
<br>
Cheers,<br>
Nick.<br>
<br>
--<br>
Nick Coghlan | <a href="mailto:ncoghlan@gmail.com">ncoghlan@gmail.com</a> | Brisbane, Australia<br>
<div class="HOEnZb"><div class="h5">_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/python-dev" target="_blank">http://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="http://mail.python.org/mailman/options/python-dev/greg%40krypto.org" target="_blank">http://mail.python.org/mailman/options/python-dev/greg%40krypto.org</a><br>
</div></div></blockquote></div><br></div>