[Numpy-discussion] tracing numpy data allocation with python callbacks
Thouis (Ray) Jones
thouis at gmail.com
Thu May 17 17:18:22 EDT 2012
On Thu, May 17, 2012 at 9:52 PM, Nathaniel Smith <njs at pobox.com> wrote:
> I'd be tempted to just see if I could get by with massif or another
> "real" heap profiler -- unfortunately the ones I know are C oriented,
> but might still be useful...
I got some very useful information from Fabien's technique, which led
me to wanting to know more, particularly about spikes in ephemeral
allocations, such as for the subexpressions of "a = b * c + d * e".
The main benefit to what I posted over heap profilers in general was
that it gave a way for tracing at the python level, so I can easily do
things like record the allocations (long-lived and ephemeral) by what
line in the python code call stack is responsible.
Perhaps I'll rewrite it to be cleaner, something like:
numpy.trace_data_allocations(malloc_callback=None, free_callback=None,
realloc_callback=None)
Where "None" turns off tracing of that particular call. It should be
efficient when not activated (check a single flag).
Ray Jones
More information about the NumPy-Discussion
mailing list