[Python-Dev] RFC: PEP 454: Add a new tracemalloc module

[Victor Stinner]

2013/9/4 Victor Stinner <victor.stinner at gmail.com>:
> http://www.python.org/dev/peps/pep-0454/
>
> PEP: 454
> Title: Add a new tracemalloc module to trace Python memory allocations
> Version: $Revision$
> Last-Modified: $Date$
> Author: Victor Stinner <victor.stinner at gmail.com>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 3-September-2013
> Python-Version: 3.4

I added a function get_tracemalloc_size() to see how much memory is
used by the tracemalloc module itself. Result on the Python test
suite:

* 1 frame: +52% (+%68%)
  Python=34 MiB; _tracemalloc=18 MiB, tracemalloc.py=5 MiB
* 10 frames: +155% (+170%)
  Python=34 MiB, _tracemalloc=53 MiB, tracemalloc.py=5 MiB
* 100 frames: +1273% (+1283%)
  Python=30 MiB, _tracemalloc=382 MiB, tracemalloc.py=6 MiB

On a small application and a computer with GB of memory, it may not
matter. In a big application on an embedded device, it can be a
blocker point to use tracemalloc. So I added filters (on the filename
and line number) directly in the C module:


``add_filter(include: bool, filename: str, lineno: int=None)`` function:

    Add a filter. If *include* is ``True``, only trace memory blocks
    allocated in a file with a name matching *filename*. If
    *include* is ``False``, don't trace memory blocks allocated in a
    file with a name matching *filename*.

    The match is done using *filename* as a prefix. For example,
    ``'/usr/bin/'`` only matchs files the ``/usr/bin`` directories. The
    ``.pyc`` and ``.pyo`` suffixes are automatically replaced with
    ``.py`` when matching the filename.

    *lineno* is a line number. If *lineno* is ``None`` or lesser than
    ``1``, it matches any line number.

``clear_filters()`` function:

    Reset the filter list.

``get_filters()`` function:

    Get the filters as list of
    ``(include: bool, filename: str, lineno: int)`` tuples.

    If *lineno* is ``None``, a filter matchs any line number.

   By default, the filename of the Python tracemalloc module
   (``tracemalloc.py``) is excluded.


Right now, the match is done using a PyUnicode_Tailmatch(). It is not
convinient. I will see if it is possible to implement the joker
character "*" matching any string, so the API would be closer to
Snapshot.filter_filenames() (which uses fnmatch.fnmatch).

Victor