[Python-Dev] What is the precise problem? [was: Reference cycles in Exception.__traceback__]

[Maciej Fijalkowski]

On Mon, Mar 10, 2014 at 7:49 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Mar 10, 2014 at 10:39 AM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> On Mon, Mar 10, 2014 at 7:35 PM, Guido van Rossum <guido at python.org>
>> wrote:
>> > On Mon, Mar 10, 2014 at 10:30 AM, Maciej Fijalkowski <fijall at gmail.com>
>> > wrote:
>> >>
>> >> On Mon, Mar 10, 2014 at 3:23 PM, Victor Stinner
>> >> <victor.stinner at gmail.com> wrote:
>> >> > 2014-03-10 13:11 GMT+01:00 Maciej Fijalkowski <fijall at gmail.com>:
>> >> >> It was agreed long time ago that the immediate finalization is an
>> >> >> implementation specific detail and it's not guaranteed. You should
>> >> >> not
>> >> >> rely on __del__s being called timely one way or another. Why would
>> >> >> you
>> >> >> require this for the program to work correctly in the particular
>> >> >> example of __traceback__?
>> >> >
>> >> > For asyncio, it's very useful to see unhandled exceptions as early as
>> >> > possible. Otherwise, your program is blocked and you don't know why.
>> >> >
>> >> > Guido van Rossum suggests to run gc.collect() regulary:
>> >> > http://code.google.com/p/tulip/issues/detail?id=42
>> >> >
>> >> > Victor
>> >>
>> >> twisted goes around it by attaching errback by hand. Would that work
>> >> for
>> >> tulip?
>> >
>> >
>> > Can you describe that idea in more detail?
>>
>> Essentially, instead of relying on deferred to be garbage collected,
>> you attach an errback like this:
>>
>> deferred.addErrback(callback_that_writes_to_log)
>>
>> so in case of a failure, you get a traceback directly in the callback
>> immediately, without relying on garbage collection.
>>
>> I'm sorry if I'm using twisted nomenclature here (it's also awfully
>> off-topic for python-dev), but making programs rely on refcounting
>> sounds like a bad idea for us (PyPy).
>
>
> IIUC the problem that Victor is trying to solve is what to do if nobody
> thought to attach an errback. Tulip makes a best effort to still log a
> traceback. We've found this very helpful (just as it is helpful that Python
> prints a traceback when synchronous code raises an exception and no except
> clause caught it).
>
> The best effort relies on GC. I am guessing that refcounting makes the
> traceback appear sooner, but there would be other ways to force it, like
> occasionally calling gc.collect() during idle times (presumably during busy
> times it will be called soon enough. :-)
>
> --
> --Guido van Rossum (python.org/~guido)

I agree this sounds like a solution. However I'm very skeptical about
changing details of __traceback__ and frames, just in order to make
refcounting work (since it would create something that would not work
on pypy for example).

Cheers,
fijal