[Python-Dev] Avoid formatting an error message on attribute error

[Nick Coghlan]

On 7 Nov 2013 21:34, "Victor Stinner" <victor.stinner at gmail.com> wrote:
>
> 2013/11/7 Steven D'Aprano <steve at pearwood.info>:
> > My initial instinct here was to say that sounded like premature
> > optimization, but to my surprise the overhead of generating the error
> > message is actually significant -- at least from pure Python 3.3 code.
>
> I ran a quick and dirty benchmark by replacing the error message with
None.
>
> Original:
>
> $ ./python -m timeit 'hasattr(1, "y")'
> 1000000 loops, best of 3: 0.354 usec per loop
> $ ./python -m timeit -s 's=1' 'try: s.y' 'except AttributeError: pass'
> 1000000 loops, best of 3: 0.471 usec per loop
>
> Patched:
>
> $ ./python -m timeit 'hasattr(1, "y")'
> 10000000 loops, best of 3: 0.106 usec per loop
> $ ./python -m timeit -s 's=1' 'try: s.y' 'except AttributeError: pass'
> 10000000 loops, best of 3: 0.191 usec per loop
>
> hasattr() is 3.3x faster and try/except is 2.4x faster on such micro
benchmark.
>
> > Given that, I wonder whether it would be worth coming up with a more
> > general solution to the question of lazily generating error messages
> > rather than changing AttributeError specifically.
>
> My first question is about keeping strong references to objects (type
> object for AttributeError). Is it an issue? If it is an issue, it's
> maybe better to not modify the code :-)
>
>
> Yes, the lazy formatting idea can be applied to various other
> exceptions. For example, TypeError message is usually build using
> PyErr_Format() to mention the name of the invalid type. Example:
>
>         PyErr_Format(PyExc_TypeError, "exec() arg 2 must be a dict, not
%.100s",
>                      globals->ob_type->tp_name);
>
> But it's not easy to store arbitary C types for PyUnicode_FromFormat()
> parameters. Argument types can be char*, Py_ssize_t, PyObject*, int,
> etc.
>
> I proposed to modify (first/only) AttributeError, because it is more
> common to ignore the AttributeError than other errors like TypeError.
> (TypeError or UnicodeDecodeError are usually unexpected.)
>
> >> It would be nice to only format the message on demand. The
> >> AttributeError would keep a reference to the type.
> >
> > Since only the type name is used, why keep a reference to the type
> > instead of just type.__name__?
>
> In the C language, type.__name__ does not exist, it's a char* object.
> If the type object is destroyed, type->tp_name becomes an invalid
> pointer. So AttributeError should keep a reference on the type object.
>
> >> AttributeError.args would be (type, attr) instead of (message,).
> >> ImportError was also modified to add a new "name "attribute".

The existing signature continued to be supported, though.

> >
> > I don't like changing the signature of AttributeError. I've got code
> > that raises AttributeError explicitly.
>
> The constructor may support different signature for backward
> compatibility: AttributeError(message: str) and AttributeError(type:
> type, attr: str).
>
> I'm asking if anyone relies on AttributeError.args attribute.

The bigger problem is you can't change the constructor signature in a
backwards incompatible way. You would need a new class method as an
alternative constructor instead, or else use optional parameters.

Cheers,
Nick.

>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131107/c09ef2ea/attachment.html>