Guido van Rossum wrote:
Please don't act so surprised. There are about 4 relevant PEPs: 344, 3109, 3110, 3134 (the latter replacing 344).
I knew about the discussions around chained exception. I wasn't aware of the idea to keep a traceback object on the exception object itself. PEP 3134 also mentioned the case we're currently discussing: """ Open Issue: Garbage Collection The strongest objection to this proposal has been that it creates cycles between exceptions and stack frames [12]. Collection of cyclic garbage (and therefore resource release) can be greatly delayed. >>> try: >>> 1/0 >>> except Exception, err: >>> pass will introduce a cycle from err -> traceback -> stack frame -> err, keeping all locals in the same scope alive until the next GC happens. Today, these locals would go out of scope. There is lots of code which assumes that "local" resources -- particularly open files -- will be closed quickly. If closure has to wait for the next GC, a program (which runs fine today) may run out of file handles. Making the __traceback__ attribute a weak reference would avoid the problems with cyclic garbage. Unfortunately, it would make saving the Exception for later (as unittest does) more awkward, and it would not allow as much cleanup of the sys module. A possible alternate solution, suggested by Adam Olsen, would be to instead turn the reference from the stack frame to the 'err' variable into a weak reference when the variable goes out of scope [13]. """ So obviously this case had already been discussed before. Was a solution found and implemented that addresses the problem ?
Also note that the traceback is only kept alove if the exception object is explicitly copied out of the except block that caught it -- normally the exception object is deleted when that block is left.
Right, but only if you do not use the exception object for other purposes elsewhere. If you do that a lot in your application, it appears that the only way around keeping lots of traceback objects alive is by explicitly setting .__traceback__ to None before storing away the exception object. Think of e.g. an application that does a long running calculation. Such applications typically want to continue processing even in case of errors and report all errors at the end of the run. If a programmer is unaware of the traceback issue, he'd likely run into a memory problem without really knowing where to look for the cause. Also note that garbage collection will not necessarily do what the user expects: it is well possible that big amounts of memory will stay allocated as unused space in pymalloc. This is not specific to the discussed case, but still a valid user concern. Greg Hazel observed this situation in his example.
--Guido
On Sat, Jun 26, 2010 at 2:53 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Antoine Pitrou wrote:
On Sat, 26 Jun 2010 13:03:38 +0200 "M.-A. Lemburg" <mal@egenix.com> wrote:
Greg Ewing wrote:
ghazel@gmail.com wrote:
I'm interested in a feature which allows users to discard the locals and globals references from frames held by a traceback object.
Wouldn't it be better to write safer code and not store a reference to the traceback object in the first place ?
In Python 3, tracebacks are stored as an attribute of the corresponding exception:
try: 1/0 ... except Exception as _: e = _ ... e.__traceback__ <traceback object at 0x7ff69fdbf908>
Ouch.
So you explicitly need get rid off the traceback in Python3 if you want to avoid keeping the associated objects alive during exception processing ?
I think that design decision needs to be revisited. Tracebacks are needed for error reporting, but (normally) not for managing error handling or recovery.
E.g. it is not uncommon to store exception objects in a list for later batched error reporting. With the traceback being referenced on those object and the traceback chain keeping references to all frames alive, this kind of processing won't be feasible anymore.
What's even more important is that programmers are unlikely going to be aware of this detail and its implications.
Also: What's the use case for creating traceback objects outside the Python interpreter core ?
He's not talking about creating traceback objects outside the core, but being able to reuse tracebacks created by the core without keeping alive a whole chain of objects.
With the question I was referring to the suggestion by Greg Ewing in which he seemed to imply that Pyrex and Cython create traceback objects.
It's a real need when you want to do careful error handling/reporting without wasting too many resources. As already mentioned, Twisted has a bunch of code to work around that problem, since errors can be quite long-lived in a pipelined asynchronous execution model.
With the above detail, I completely agree. In fact, more than that: I think we should make storing the traceback in exception.__traceback__ optional and not the default, much like .__context__ and .__cause__.
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jun 26 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 22 days to go
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 27 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 21 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/