[Python-Dev] cpython: Issue #3329: Add new APIs to customize memory allocators

[Antoine Pitrou]

On Sun, 16 Jun 2013 02:18:32 +0200
Victor Stinner <victor.stinner at gmail.com> wrote:
> 2013/6/15 Antoine Pitrou <solipsis at pitrou.net>:
> > On Sat, 15 Jun 2013 03:54:50 +0200
> > Victor Stinner <victor.stinner at gmail.com> wrote:
> >> The addition of PyMem_RawMalloc() is motivated by the issue #18203
> >> (Replace calls to malloc() with PyMem_Malloc()). The goal is to be
> >> able to setup a custom allocator for *all* allocation made by Python,
> >> so malloc() should not be called directly. PyMem_RawMalloc() is
> >> required in places where the GIL is not held (ex: in os.getcwd() on
> >> Windows).
> >
> > We already had this discussion on IRC and this argument isn't very
> > convincing to me. If os.getcwd() doesn't hold the GIL while allocating
> > memory, then you should fix it to hold the GIL while allocating memory.
> 
> The GIL is released for best performances, holding the GIL would have
> an impact on performances.

Well, do you have benchmark numbers? Do you have a workload where
getcwd() is performance-critical to the point that a single
GIL-protected allocation may slow down your program?

> "Private" functions can be used by applications, it's just that Python
> doesn't give any backward compatibility warranty. Am I right?

Anyone "can" use anything obviously, but when it's private, it can be
changed or removed in any release.
If the only goal for these functions is to be used by applications,
though, it's quite a bad idea to make them private.

Regards

Antoine.