[Python-ideas] feature to make traceback objects usable without references to frame locals and globals
mal at egenix.com
Sun Jun 27 19:20:07 CEST 2010
Guido van Rossum wrote:
> On Sat, Jun 26, 2010 at 4:45 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> Also note that garbage collection will not necessarily do what
>> the user expects: it is well possible that big amounts of
>> memory will stay allocated as unused space in pymalloc.
>> This is not specific to the discussed case, but still a valid
>> user concern. Greg Hazel observed this situation in his
> Aha. So whereas the process size ballooned, there is no actual memory
> leak (his example threw away the exception each time through the
> loop), it's just that looking at process size is a bad way to assess
> memory leaks. I would like to reject this then as "that's just how
> Python's memory allocation works". As you say, it's not specific to
> this case; it comes up occasionally and it's just a matter of user
pymalloc has gotten a lot better since it was fixed in Python 2.5
to return unused chunks of memory to the OS, but we still have the
issue of fragmented arenas with cases of just a few bytes
keeping 256kB (the size of an arena) allocated.
> I don't think anything should be done about __traceback__ either --
> frameworks that have this problem can work around it in various ways.
> Or, at least I don't see a reason to panic and roll back the feature.
> Maybe eventually it can be improved by adding some kind of
> functionality to control some details of the behavior.
Not necessarily roll back the feature, but an implementation
that deliberately introduces circular references is not really
Since tracebacks on exceptions are rarely used by applications,
I think it would be better to turn them into weak references.
The arguments against doing this in the PEP appear rather
weak compared to the potential issue for non-expert Python
Making the __traceback__ attribute a weak reference would avoid the
problems with cyclic garbage. Unfortunately, it would make saving
the Exception for later (as unittest does) more awkward, and it would
not allow as much cleanup of the sys module.
Special use cases that want to save the traceback for later use
can always explicitly convert the traceback into a real (non-weak)
reference. I don't understand the reference to the sys module
cleanup, so can't comment on that.
Professional Python Services directly from the Source (#1, Jun 27 2010)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 21 days to go
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
More information about the Python-ideas