[Python-checkins] peps: PEP 454: Add a new tracemalloc module to trace Python memory allocations
victor.stinner
python-checkins at python.org
Tue Sep 3 00:36:12 CEST 2013
http://hg.python.org/peps/rev/d864f728f5da
changeset: 5089:d864f728f5da
user: Victor Stinner <victor.stinner at gmail.com>
date: Tue Sep 03 00:35:54 2013 +0200
summary:
PEP 454: Add a new tracemalloc module to trace Python memory allocations
files:
pep-0454.txt | 257 +++++++++++++++++++++++++++++++++++++++
1 files changed, 257 insertions(+), 0 deletions(-)
diff --git a/pep-0454.txt b/pep-0454.txt
new file mode 100644
--- /dev/null
+++ b/pep-0454.txt
@@ -0,0 +1,257 @@
+PEP: 454
+Title: Add a new tracemalloc module to trace Python memory allocations
+Version: $Revision$
+Last-Modified: $Date$
+Author: Victor Stinner <victor.stinner at gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 3-September-2013
+Python-Version: 3.4
+
+
+Abstract
+========
+
+Add a new tracemalloc module to trace Python memory allocations.
+
+
+
+Rationale
+=========
+
+This PEP proposes to a new ``tracemalloc`` module, a debug tool to trace
+memory allocations made by Python. The module provides the following
+information:
+
+* Statistics on allocations per Python line number (file and line):
+ size, number, and average size of allocations
+* Compute delta between two "snapshots"
+* Location of a memory allocation: Python filename and line number
+
+To trace the most Python memory allocations, the module should be
+enabled as early as possible in your application by calling
+``tracemalloc.enable()`` function, by setting the ``PYTHONTRACEMALLOC``
+environment variable to ``1``, or by using ``-X tracemalloc`` command
+line option.
+
+
+API
+===
+
+Functions
+---------
+
+enable():
+
+ Start tracing Python memory allocations.
+
+disable():
+
+ Stop tracing Python memory allocations and stop the timer started by
+ ``start_timer()``.
+
+is_enabled():
+
+ ``True`` if the module is enabled, ``False`` otherwise.
+
+get_object_trace(obj):
+
+ Get the trace of a Python object *obj* as a namedtuple with 3 attributes:
+
+ - ``size``: size in bytes of the object
+ - ``filename``: name of the Python script where the object was allocated
+ - ``lineno``: line number where the object was allocated
+
+ Return ``None`` if tracemalloc did not save the location where the object
+ was allocated, for example if tracemalloc was disabled.
+
+get_process_memory():
+
+ Get the memory usage of the current process as a meminfo namedtuple with
+ two attributes:
+
+ * ``rss``: Resident Set Size in bytes
+ * ``vms``: size of the virtual memory in bytes
+
+ Return ``None`` if the platform is not supported.
+
+ Use the ``psutil`` module if available.
+
+start_timer(delay: int, func: callable, args: tuple=(), kwargs: dict={}):
+
+ Start a timer calling ``func(*args, **kwargs)`` every *delay*
+ seconds.
+
+ The timer is based on the Python memory allocator, it is not real
+ time. *func* is called after at least *delay* seconds, it is not
+ called exactly after *delay* seconds if no Python memory allocation
+ occurred.
+
+ If ``start_timer()`` is called twice, previous parameters are
+ replaced. The timer has a resolution of 1 second.
+
+ `start_timer()`` is used by ``DisplayTop`` and ``TakeSnapshot``
+ to run regulary a task.
+
+stop_timer():
+
+ Stop the timer started by ``start_timer()``.
+
+
+DisplayTop
+----------
+
+DisplayTop(count: int, file=sys.stdout) class:
+
+ Display the list of the N biggest memory allocations.
+
+display():
+
+ Display the top
+
+start(delay: int):
+
+ Start a task using tracemalloc timer to display the top every delay seconds.
+
+stop():
+
+ Stop the task started by the ``DisplayTop.start()`` method
+
+color attribute:
+
+ Use color if True (bool, default: stream.isatty()).
+
+compare_with_previous attribute:
+
+ If ``True``, compare with the previous top if ``True``. If ``False``,
+ compare with the first one (bool, default: ``True``): .
+
+filename_parts attribute:
+
+ Number of displayed filename parts (int, default: ``3``).
+
+show_average attribute:
+
+ If ``True``, show the average size of allocations (bool, default: ``True``).
+
+show_count attribute:
+
+ If ``True``, show the number of allocations (bool, default: ``True``).
+
+show_lineno attribute:
+
+ If ``True``, use also the line number, not only the filename (bool, default:
+ ``True``). If ``False``, only show the filename.
+
+show_size attribute:
+
+ if ``True``, show the size of allocations (bool, default: ``True``).
+
+user_data_callback attribute:
+
+ Optional callback collecting user data (callable, default:
+ ``None``). See ``Snapshot.create()``.
+
+
+Snapshot
+--------
+
+Snapshot() class:
+
+ Snapshot of Python memory allocations. Use ``TakeSnapshot`` to regulary
+ take snapshots.
+
+create(user_data_callback=None)
+
+ Take a snapshot. If user_data_callback is specified, it must be a callable
+ object returning a list of (title: str, format: str, value: int). format
+ must be "size". The list must always have the same length and the same order
+ to be able to compute differences between values.
+
+ Example: ``[('Video memory', 'size', 234902)]``.
+
+filter_filenames(patterns: str|list, include: bool)
+
+ Remove filenames not matching any pattern if include is True, or remove
+ filenames matching a pattern if include is False (exclude). See
+ fnmatch.fnmatch() for the syntax of patterns.
+
+write(filename)
+
+ Write the snapshot into a file.
+
+pid attribute:
+
+ Identifier of the process which created the snapshot (int).
+
+stats attribute:
+
+ Raw memory allocation statistics (dict).
+
+timestamp attribute:
+
+ Date and time of the creation of the snapshot (str).
+
+
+TakeSnapshot
+------------
+
+TakeSnapshot class:
+
+ Task taking snapshots of Python memory allocations: write them into files.
+
+start(delay: int)
+
+ Start a task taking a snapshot every delay seconds.
+
+stop():
+
+ Stop the task started by the ``TakeSnapshot.start()`` method.
+
+take_snapshot():
+
+ Take a snapshot.
+
+filename_template attribute:
+
+ Template (str) to create a filename. "Variables" can be used in the
+ template:
+
+ * ``$pid``: identifier of the current process
+ * ``$timestamp``: current date and time
+ * ``$counter``: counter starting at 1 and incremented at each snapshot
+
+user_data_callback attribute:
+
+ Optional callback collecting user data (callable, default: None).
+ See ``Snapshot.create()``.
+
+
+Links
+=====
+
+* `Python issue #18874: Add a new tracemalloc module to trace Python
+ memory allocations <http://bugs.python.org/issue18874>`_
+
+Similar projects:
+
+* `Meliae: Python Memory Usage Analyzer
+ <https://pypi.python.org/pypi/meliae>`_
+* `Issue #3329: API for setting the memory allocator used by Python
+ <http://bugs.python.org/issue3329>`_
+* `Guppy-PE: umbrella package combining Heapy and GSL
+ <http://guppy-pe.sourceforge.net/>`_
+* `PySizer <http://pysizer.8325.org/>`_: developed for Python 2.4
+* `memory_profiler <https://pypi.python.org/pypi/memory_profiler>`_
+* `pympler <http://code.google.com/p/pympler/>`_
+* `Dozer <https://pypi.python.org/pypi/Dozer>`_: WSGI Middleware version of
+ the CherryPy memory leak debugger
+* `objgraph <http://mg.pov.lt/objgraph/>`_
+* `caulk <https://github.com/smartfile/caulk/>`_
+
+Copyright
+=========
+
+This document has been placed into the public domain.
+
--
Repository URL: http://hg.python.org/peps
More information about the Python-checkins
mailing list