[Python-Dev] lookdict

[Vladimir Marangozov]

Aha. Thanks for the explanation.

Guido van Rossum wrote:
> 
> Thanks, Marc-Andre, for pointing out that Fred's lookdict code is
> actually an improvement.

Right. I was too fast. There is some speedup due to the string
specialization. I'll post a patch to SF with some more tweaks
of this implementation. Briefly:

- do not call PyErr_Clear() systematically after PyObject_Compare();
  only if (!error_restore && PyErr_Occurred())
- defer variable initializations after common return cases
- avoid using more vars in lookdict_string + specialize string_compare()
- inline the most frequest case in PyDict_GetItem (the first item probe)

> The reason for all this is that we found that lookdict() calls
> PyObject_Compare() without checking for errors.  If there's a key that
> raises an error when compared to another key, the keys compare unequal
> and an exception is set, which may disturb an exception that the
> caller of PyDict_GetItem() might be calling.  PyDict_GetItem() is
> documented as never raising an exception.  This is actually not strong
> enough; it was actually intended to never clear an exception either.
> The potential errors from PyObject_Compare() violate this contract.
> Note that these errors are nothing new; PyObject_Compare() has been
> able to raise exceptions for a long time, e.g. from errors raised by
> __cmp__().
> 
> The first-order fix is to call PyErr_Fetch() and PyErr_restore()
> around the calls to PyObject_Compare().  This is slow (for reasons
> Vladimir points out) even though Fred was very careful to only call
> PyErr_Fetch() or PyErr_Restore() when absolutely necessary and only
> once per lookdict call.  The second-order fix therefore is Fred's
> specialization for string-keys-only dicts.
> 
> There's another problem: as fixed, lookdict needs a current thread
> state!  (Because the exception state is stored per thread.)  There are
> cases where PyDict_GetItem() is called when there's no thread state!
> The first one we found was Tim Peters' patch for _PyPclose (see
> separate message).  There may be others -- we'll have to fix these
> when we find them (probably after 2.0b1 is released but hopefully
> before 2.0 final).

Hm. Question: is it possible for the thread state to swap during
PyObject_Compare()? If it is possible, things are more complicated
than I thought...

-- 
       Vladimir MARANGOZOV          | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252