[Python-Dev] with_traceback

[Andrew Dalke]

PJE:
> Then don't do that, as it's bad style for Python 3.x.  ;-)

It's bad style for 3.x only if Python goes with this interface.  If
it stays with the 2.x style then there's no problem.  There
may also be solutions which are cleaner and which don't
mutate the exception instance.

I am not proposing such a syntax.  I have ideas I am not a
language designer and have long given up the idea that I
might be good at it.

> This does mean you won't be able to port your code to 3.x style until
> you've gotten rid of shared exception instances from all your dependencies,
> but 3.x porting requires all your dependencies to be ported anyway.

What can be done to minimize the number of dependencies which
need to be changed?

> It should be sufficient in both 2.x and 3.x for with_traceback() to raise
> an error if the exception already has a traceback -- this should catch any
> exception instance reuse.

That would cause a problem in my example where I save then
reraise the exception, as

  raise saved_err.with_traceback(saved_err.__traceback__)

> >What is the correct way to rewrite this for use
> >with "with_traceback"?  Is it
  [...]

> No, it's more like this:
>
>      try:
>          for dirname in ...
>              try:
>                  return ...
>              except Exception as err:
>                 saved_err = err
>          raise saved_err
>      finally:
>          del saved_err

I don't get it.  The "saved_err" has a __traceback__
attached to it, and is reraised.  Hence it gets the old
stack, right?

Suppose I wrote

ERR = Exception("Do not do that")

try:
  f(x)
except Exception:
  raise ERR

try:
  f(x*2)
except Exception:
  raise ERR

Yes it's bad style, but people will write it.  The ERR gets
the traceback from the first time there's an error, and
that traceback is locked in ... since raise won't change
the __traceback__ if one exists.  (Based on what you
said it does.)


> I've added the outer try-finally block to minimize the GC impact of the
> *original* code you showed, as the `saved_tb` would otherwise have created
> a cycle.  That is, the addition is not because of the porting, it's just
> something that you should've had to start with.

Like I said, I used code based on os._execvpe.  Here's the code

    saved_exc = None
    saved_tb = None
    for dir in PATH:
        fullname = path.join(dir, file)
        try:
            func(fullname, *argrest)
        except error, e:
            tb = sys.exc_info()[2]
            if (e.errno != ENOENT and e.errno != ENOTDIR
                and saved_exc is None):
                saved_exc = e
                saved_tb = tb
    if saved_exc:
        raise error, saved_exc, saved_tb
    raise error, e, tb

I see similar use in atexit._run_exitfuncs, though as Python
is about to exit it won't make a real difference.

doctest shows code like

         >>> exc_info = failure.exc_info
         >>> raise exc_info[0], exc_info[1], exc_info[2]

SimpleXMLRPCServer does things like

        except:
            # report exception back to server
            exc_type, exc_value, exc_tb = sys.exc_info()
            response = xmlrpclib.dumps(
                xmlrpclib.Fault(1, "%s:%s" % (exc_type, exc_value)),
                encoding=self.encoding, allow_none=self.allow_none,
                )


I see threading.py gets it correctly.

My point here is that most Python code which uses the traceback
term doesn't break the cycle, so must be caught by the gc.  While
there might be a more correct way to do it, it's too complicated
for most to get it right.

> Anyway, the point here is that in 3.x style, most uses of 3-argument raise
> just disappear altogether.  If you hold on to an exception instance, you
> have to be careful about it for GC, but no more so than in current Python.

Where people already make a lot of mistakes.  But my concern
is not in the gc, it's in the mutability of the exception causing hard
to track down problems in code which is written by beginning to
intermediate users.

> The "save one instance and use it forever" use case is new to me - I've
> never seen nor written code that uses it before now.  It's definitely
> incompatible with 3.x style, though.

I pointed out an example in pyparsing.  Thomas W. says he's
seen other code.  I've been looking for another real example but
as this is relatively uncommon code, I don't have a wide enough
corpus for the search.  I also don't know of a good tool for searching
for this sort of thing.  (Eg, www.koders.com doesn't help.)

It's a low probability occurance.  So is the use of the 3 arg raise.
Hence it's hard to get good intuition about problems which might
arise.

        Andrew
        dalke at dalkescientific.com