[Python-checkins] peps: PEP 445: more background

victor.stinner python-checkins at python.org
Sun Jun 16 03:49:42 CEST 2013

changeset:   4932:5e26a6276514
user:        Victor Stinner <victor.stinner at gmail.com>
date:        Sun Jun 16 03:49:29 2013 +0200
  PEP 445: more background

  pep-0445.txt |  59 +++++++++++++++++++++++++++++++++++++++-
  1 files changed, 58 insertions(+), 1 deletions(-)

diff --git a/pep-0445.txt b/pep-0445.txt
--- a/pep-0445.txt
+++ b/pep-0445.txt
@@ -151,13 +151,65 @@
 XXX To be done (Kristján Valur Jónsson) XXX
+External libraries
+* glib: `g_mem_set_vtable()
+  <http://developer.gnome.org/glib/unstable/glib-Memory-Allocation.html#g-mem-set-vtable>`_
+Memory allocators
+The C standard library provides the well known ``malloc()`` function. Its
+implementation depends on the platform and of the C library. The GNU C library
+uses a modified ptmalloc2, based on "Doug Lea's Malloc" (dlmalloc). FreeBSD
+uses `jemalloc <http://www.canonware.com/jemalloc/>`_. Google provides
+tcmalloc which is part of `gperftools <http://code.google.com/p/gperftools/>`_.
+``malloc()`` uses two kinds of memory: heap and memory mappings. Memory
+mappings are usually used for large allocations (ex: larger than 256 KB),
+whereas the heap is used for small allocations.
+The heap is handled by ``brk()`` and ``sbrk()`` system calls on Linux, and is
+contiguous.  Memory mappings are handled by ``mmap()`` on UNIX and
+``VirtualAlloc()`` on Windows, they are discontiguous. Releasing a memory
+mapping gives back the memory immediatly to the system. For the heap, memory is
+only gave back to the system if it is at the end of the heap. Otherwise, the
+memory will only gave back to the system when all the memory located after the
+released memory are also released. This limitation causes an issue called the
+"memory fragmentation": the memory usage seen by the system may be much higher
+than real usage.
+Windows provides a `Low-fragmentation Heap
+The Linux kernel uses `slab allocation
+The glib library has a `Memory Slice API
+efficient way to allocate groups of equal-sized chunks of memory
-Memory allocators:
+CPython issues related to memory allocation:
 * `Issue #3329: Add new APIs to customize memory allocators
+* `Issue #13483: Use VirtualAlloc to allocate memory arenas
+  <http://bugs.python.org/issue13483>`_
+* `Issue #16742: PyOS_Readline drops GIL and calls PyOS_StdioReadline, which
+  isn't thread safe <http://bugs.python.org/issue16742>`_
+* `Issue #18203: Replace calls to malloc() with PyMem_Malloc()
+  <http://bugs.python.org/issue18203>`_
+* `Issue #18227: Use Python memory allocators in external libraries like zlib
+  or OpenSSL <http://bugs.python.org/issue18227>`_
+Projects analyzing the memory usage of Python applications:
 * `pytracemalloc
 * `Meliae: Python Memory Usage Analyzer
@@ -167,6 +219,11 @@
 * `PySizer (developed for Python 2.4)
+APIs to set a custom memory allocator and/or hook memory allocators:
+* `GNU libc: Memory Allocation Hooks
+  <http://www.gnu.org/software/libc/manual/html_node/Hooks-for-Malloc.html>`_
 * `Python benchmark suite

Repository URL: http://hg.python.org/peps

More information about the Python-checkins mailing list