[Cython] Bug: _PyType_Lookup shortcut in Cython 0.20 breaks lookup of __exit__ in Python 2.6

[Stefan Behnel]

[forwarding to cython-devel]

Wouter Bolsterlee, 02.02.2014 16:40:
> Cython 0.20 introduced __Pyx_PyObject_LookupSpecial as a shortcut to
> lookup special methods. The commit that introduced this change is
> 8f6412275c4c2d1dcf43ad40306f858b114104ed and can be seen here:
> 
> https://github.com/cython/cython/commit/8f6412275c4c2d1dcf43ad40306f858b114104ed
>         
> It seems this change does not work with Python 2.6. The newly introduced
> function calls _PyType_Lookup(), which fails to lookup (at least)
> __exit__ (on a threading.Lock) instance when run under Python 2.6. The
> exact call is here:
> 
> https://github.com/cython/cython/commit/8f6412275c4c2d1dcf43ad40306f858b114104ed#diff-83083bdce5dcca284394ead83b8d3f99R1053
>         
> For my Plyvel project (https://plyvel.readthedocs.org/ and
> https://github.com/wbolster/plyvel) this issue manifests as follows:
> 
> $ cython --version
>         Cython version 0.20
>         
>         $ python 
>         Python 2.6.8 (unknown, Jan 26 2013, 14:35:25) 
>         [GCC 4.7.2] on linux2
>         Type "help", "copyright", "credits" or "license" for more
> information.
>         >>> import plyvel
>         >>> db = plyvel.DB('/tmp/testdb', create_if_missing=True)
>         >>> db.close()
>         Traceback (most recent call last):
>           File "<stdin>", line 1, in <module>
>           File "_plyvel.pyx", line 246, in plyvel._plyvel.DB.close
> (plyvel/_plyvel.cpp:3563)
>           File "_plyvel.pyx", line 252, in plyvel._plyvel.DB.close
> (plyvel/_plyvel.cpp:3221)
>         AttributeError: __exit__
>         >>> 
>         Exception AttributeError: '__exit__' in
> 'plyvel._plyvel.DB.__dealloc__' ignored
>         
> The same Cython source code works fine against Python 2.7, 3.2, and 3.3.
> Reverting to Cython 0.19 makes the code run correctly on Python 2.6,
> 2.7, 3.2, and 3.3 again. This means only the combination of Python 2.6
> and Cython 0.20 fails.
> 
> To check for yourself, you can run "make test" on a Git clone of Plyvel
> with either Cython 0.19 or Cython 0.20 installed, and spot the
> differences. Or run "tox" to automatically test against multiple Python
> versions.

Ok, I don't know what the exact difference is, but I do believe you that
Py2.6 doesn't play as nicely here as Py3. In any case, the CPython code
that I adapted it from didn't actually exist in Py2.6/3.1, so the
guarantees don't need to be there either.

Given that this is essentially an optimisation (and otherwise meant to
*improve* the compatibility with CPython's own behaviour), I'll disable
this in Py2.6 and Py3.1 for the next bug fix release.

https://github.com/cython/cython/commit/b8e37bc15373f303691de2d256ad99045221c9e0

It would be good if you could come up with a failing regression test case
for this, so that we can make sure it actually keeps working in the future.


> Please CC me when responding, because I'm not subscribed to
> cython-devel.

cython-devel is actually fairly low traffic, so temporarily subscribing to
it won't hurt much.

Stefan