[Python-Dev] lookdict

[Guido van Rossum]

Thanks, Marc-Andre, for pointing out that Fred's lookdict code is
actually an improvement.

The reason for all this is that we found that lookdict() calls
PyObject_Compare() without checking for errors.  If there's a key that
raises an error when compared to another key, the keys compare unequal
and an exception is set, which may disturb an exception that the
caller of PyDict_GetItem() might be calling.  PyDict_GetItem() is
documented as never raising an exception.  This is actually not strong
enough; it was actually intended to never clear an exception either.
The potential errors from PyObject_Compare() violate this contract.
Note that these errors are nothing new; PyObject_Compare() has been
able to raise exceptions for a long time, e.g. from errors raised by
__cmp__().

The first-order fix is to call PyErr_Fetch() and PyErr_restore()
around the calls to PyObject_Compare().  This is slow (for reasons
Vladimir points out) even though Fred was very careful to only call
PyErr_Fetch() or PyErr_Restore() when absolutely necessary and only
once per lookdict call.  The second-order fix therefore is Fred's
specialization for string-keys-only dicts.

There's another problem: as fixed, lookdict needs a current thread
state!  (Because the exception state is stored per thread.)  There are
cases where PyDict_GetItem() is called when there's no thread state!
The first one we found was Tim Peters' patch for _PyPclose (see
separate message).  There may be others -- we'll have to fix these
when we find them (probably after 2.0b1 is released but hopefully
before 2.0 final).

--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)