[Python-Dev] PEP 454 (tracemalloc) disable ==> clear?

[Victor Stinner]

Le 30 oct. 2013 20:58, "Jim Jewett" <jimjjewett at gmail.com> a écrit :
> hough if you use a dict internally, that might not
> be the case.

Tracemalloc uses a {address: trace} duct internally.

>  If you return it as a list instead of a dict, but that list is
> NOT in time-order, that is worth documenting

Ok i will document it.

> Also, am I misreading the documentation of get_traces() function?
>
>     Get traces of memory blocks allocated by Python.
>     Return a list of (size: int, traceback: tuple) tuples.
>     traceback is a tuple of (filename: str, lineno: int) tuples.
>
>
> So it now sounds like you don't bother to emit de-allocation
> events because you just remove the allocation from your
> internal data structure.

I don't understand your question. Tracemalloc does not store events but
traces. When a memory block is deallocated, it us removed from the internal
dict (and so from get_traces() list).

> I still don't see anything here(*) that requires even saving
> the address, let alone preventing re-use.

The address must be stored internally to maintain the internal dict. See
the C code.

> (1)  Whoa -- memory hog!  How can I fix this?
>
> (2)  I know -- track allocallocations, with a traceback showing why they
> were made.  (At a minimum, I would like to be able to subclass your
> tool to do this -- preferably without also keeping the full history in
> memory.)

What do you mean by "full history" and "subclass your tool"?

> (3)  Oh, maybe I should skip the ones that really are temporary and
> get cleaned up.  (You make this easy by handling the de-allocs,
> though I'm not sure those events get exposed to anyone working at
> the python level, as opposed to modifying and re-compiling.)

If your temporary objects are destroyed before you call get_traces(), you
will not see them in get_traces(). I don't understand.

> (4)  hmm... still too big ... I should use filters.  (But will changing
those
> filters while tracing is enabled mess up your current implementation?)

If you call add_filter(), new traces() will be filtered. Not the old ones,
as explained in the doc. What do you mean by "mess up"?

> (5)  Argh.  What I really want is to know what gets allocated at times
> like XXX.
> I can do that if times-like-XXX only ever occur once per process.  I
*might* be
> able to do it with filters.  But I would rather do it by saying "trace
on" and
> "trace off".   Maybe even with a context manager around the suspicious
> places.

I don't understand "times like XXX", what is it?

To see what happened between two lines of code, you can compare two
snapshots. No need to disable tracing.

> (6)  Then, at the end of the run, I would say "give me the info about how
much
> was allocated when tracing was on."  Some of that might be going away
> again when tracing is off, but at least I know what is making the
allocations
> in the first place.  And I know that they're sticking around "long
enough".

I think you musunderstood how tracemalloc works. You should compile it and
play with it. In my opinion, you already have everything in tracemalloc for
you scenario.

> Under your current proposal, step (5) turns into
>
>     set filters
>     trace on
>     ...
>     get_traces
>     serialize to some other storage
>     trace off

s1=take_snapshot()
...
s2=take_snapshot()
...
diff=s2.statistics("lines", compare_to=s1)

> why even have
> get_traces,
> as opposed to just take_snapshot?  Is there some difference between them,
> except that a snapshot has some convenience methods and some simple
> metadata?

See the doc: Snapshot.traces is the result of get_traces().

get_traces() is here is you want to write your own tool without Snapshot.

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131030/ce963770/attachment.html>