Strange bug only happens with Python 2.2
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1. Ok, here's the story: I've had a bug report against our pyPgSQL database interface package that retrieving Large Objects doesn't work with Python 2.2.1. The reproducible traceback we get is: Traceback (most recent call last): File "p.py", line 20, in ? res = cs.fetchone() File "pyPgSQL/PgSQL.py", line 2672, in fetchone return self.__fetchOneRow() File "pyPgSQL/PgSQL.py", line 2281, in __fetchOneRow for _i in range(self.res.nfields): AttributeError: 'str' object has no attribute '__bases__' This traceback is quite obviously bogus, as self.res.nfields is a Python int and no strings are involved here whatsoever. After some debugging, I found that something very strange happens in a function call that happens in this for loop. Inside the for loop, a function typecast is called, which has this code within: if isinstance(value, PgBytea) or type(value) is PgLargeObjectType: This code is causing the problems which result in the bogus traceback later on. Now in my case, 'value' is of type PgLargeObjectType, which is a custom type from our extension module. PgBytea is a Python class. Now comes the first very strange observation: Swapping the checks, so that the 'type(value) is PgLargeObjectType' check comes first makes the problem go away. So my conclusion is that there's some problem with isinstance and my custom extension type. The second strange thing is that this only happens on Python 2.2.1 (Linux, FreeBSD, Windows), but _not_ on Python 2.1.3 or Python 2.3-CVS. Oh, the problem isn't tied to isinstance(value, PgBytea). Any isinstance check causes it later on. Of course I'm suspecting that there's some problem with the extension type. Looks like some internal interpreter data gets corrupted. No idea how to debug that, too. Does anybody have any tips where to look or how to debug this further? -- Gerhard
On Fri, 27 Sep 2002, Gerhard Häring wrote:
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1.
Guessing, but the (Jeremy's?) changes I recently backported to classobject.c on the release22-maint branch might relate to this. Can you try with a 222 build?
Ok, here's the story:
I've had a bug report against our pyPgSQL database interface package that retrieving Large Objects doesn't work with Python 2.2.1. The reproducible traceback we get is:
Traceback (most recent call last): File "p.py", line 20, in ? res = cs.fetchone() File "pyPgSQL/PgSQL.py", line 2672, in fetchone return self.__fetchOneRow() File "pyPgSQL/PgSQL.py", line 2281, in __fetchOneRow for _i in range(self.res.nfields): AttributeError: 'str' object has no attribute '__bases__'
This traceback is quite obviously bogus, as self.res.nfields is a Python int and no strings are involved here whatsoever. After some debugging, I found that something very strange happens in a function call that happens in this for loop. Inside the for loop, a function typecast is called, which has this code within:
if isinstance(value, PgBytea) or type(value) is PgLargeObjectType:
This code is causing the problems which result in the bogus traceback later on.
So something's setting an exception and not letting the interpreter know.
Now in my case, 'value' is of type PgLargeObjectType, which is a custom type from our extension module. PgBytea is a Python class.
Now comes the first very strange observation: Swapping the checks, so that the 'type(value) is PgLargeObjectType' check comes first makes the problem go away. So my conclusion is that there's some problem with isinstance and my custom extension type.
The second strange thing is that this only happens on Python 2.2.1 (Linux, FreeBSD, Windows), but _not_ on Python 2.1.3 or Python 2.3-CVS.
This is no surprise.
Oh, the problem isn't tied to isinstance(value, PgBytea). Any isinstance check causes it later on.
Huh?
Of course I'm suspecting that there's some problem with the extension type. Looks like some internal interpreter data gets corrupted. No idea how to debug that, too.
Does anybody have any tips where to look or how to debug this further?
Try a release22-maint build? Cheers, M.
* Michael Hudson <mwh@python.net> [2002-09-27 16:39 +0100]:
On Fri, 27 Sep 2002, Gerhard Häring wrote:
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1.
Guessing, but the (Jeremy's?) changes I recently backported to classobject.c on the release22-maint branch might relate to this.
Can you try with a 222 build?
Yep. The problem goes away with release22-maint :-)
Ok, here's the story: [bogus traceback, caused by:] if isinstance(value, PgBytea) or type(value) is PgLargeObjectType:
So something's setting an exception and not letting the interpreter know.
Oh, the problem isn't tied to isinstance(value, PgBytea). Any isinstance check causes it later on.
Huh?
To clarify, any isinstance(value, x), where x is a Python class, causes the problem.
[Any tips?] Try a release22-maint build?
That fixes the problem, so I'm now pretty confident it is a Python 2.2.1 problem (haven't tried with 2.2.0 yet, would that be of any use?). -- Gerhard
That fixes the problem, so I'm now pretty confident it is a Python 2.2.1 problem (haven't tried with 2.2.0 yet, would that be of any use?).
No. If it's really fixed in 2.2.2, there's nothing else we can do. But I'm curious what caused this. Can you show self-contained example code (not using SQL) that shows this behavior in 2.2.1? --Guido van Rossum (home page: http://www.python.org/~guido/)
* Michael Hudson <mwh@python.net> [2002-09-27 16:39 +0100]:
On Fri, 27 Sep 2002, Gerhard Häring wrote:
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1.
Guessing, but the (Jeremy's?) changes I recently backported to classobject.c on the release22-maint branch might relate to this.
Maybe. I've not viewed the control flow in a debugger, but my tries to come up with a minimalistic test case and my gut feeling says that this piece of code has something to do with it: static PyObject *PgLargeObject_getattr(PgLargeObject *self, char* attr) { PyObject *res; res = Py_FindMethod(PgLargeObject_methods, (PyObject *)self, attr); if (res != NULL) return res; PyErr_Clear(); if (strcmp(attr, "closed") == 0) return Py_BuildValue("l", (long)(self->lo_fd == -1)); if (!strcmp(attr, "__module__")) return Py_BuildValue("s", MODULE_NAME); if (!strcmp(attr, "__class__")) { printf("__class__ accessed!\n"); return Py_BuildValue("s", self->ob_type->tp_name); } return PyMember_Get((char *)self, PgLargeObject_members, attr); } from which I can see that isinstance tries to access the __class__ attribute. Am I supposed to /not/ provide a __class__ attribute for classic types? I haven't looked into the python22-maint changelogs yet, but I couldn't find any related registered SF bug. -- Gerhard
* Gerhard Häring <gerhard.haering@gmx.de> [2002-09-28 00:07 +0200]:
* Michael Hudson <mwh@python.net> [2002-09-27 16:39 +0100]:
On Fri, 27 Sep 2002, Gerhard Häring wrote:
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1.
Guessing, but the (Jeremy's?) changes I recently backported to classobject.c on the release22-maint branch might relate to this.
Maybe. I've not viewed the control flow in a debugger, but my tries to come up with a minimalistic test case and my gut feeling says that this piece of code has something to do with it:
static PyObject *PgLargeObject_getattr(PgLargeObject *self, char* attr) { PyObject *res;
res = Py_FindMethod(PgLargeObject_methods, (PyObject *)self, attr); if (res != NULL) return res; PyErr_Clear();
if (strcmp(attr, "closed") == 0) return Py_BuildValue("l", (long)(self->lo_fd == -1));
if (!strcmp(attr, "__module__")) return Py_BuildValue("s", MODULE_NAME);
if (!strcmp(attr, "__class__")) { printf("__class__ accessed!\n"); return Py_BuildValue("s", self->ob_type->tp_name); }
return PyMember_Get((char *)self, PgLargeObject_members, attr); }
from which I can see that isinstance tries to access the __class__ attribute. Am I supposed to /not/ provide a __class__ attribute for classic types?
I haven't looked into the python22-maint changelogs yet, but I couldn't find any related registered SF bug.
Ok, I've now further narrowed down this isinstance issue: python22-maint ==> bug does not appear python22-maint with abstract.c from Python 2.2.1 ==> bug appears So for what it's worth (i. e. not much), I'd say please /do/ include the abstract.c changes into the upcoming Python 2.2.2 :-) -- Gerhard
Ok, I've now further narrowed down this isinstance issue:
python22-maint ==> bug does not appear python22-maint with abstract.c from Python 2.2.1 ==> bug appears
So for what it's worth (i. e. not much), I'd say please /do/ include the abstract.c changes into the upcoming Python 2.2.2 :-)
I'm sure it's this change, which was backported to the 2.2 maintenance branch (and hence will be in 2.2.2). It fixes several several occurrences where an error is not cleared. ---------------------------- revision 2.101 date: 2002/04/23 22:45:44; author: bwarsaw; state: Exp; lines: +46 -9 abstract_get_bases(): Clarify exactly what the return values and states can be for this function, and ensure that only AttributeErrors are masked. Any other exception raised via the equivalent of getattr(cls, '__bases__') should be propagated up. abstract_issubclass(): If abstract_get_bases() returns NULL, we must call PyErr_Occurred() to see if an exception is being propagated, and return -1 or 0 as appropriate. This is the specific fix for a problem whereby if getattr(derived, '__bases__') raised an exception, an "undetected error" would occur (under a debug build). This nasty situation was uncovered when writing a security proxy extension type for the Zope3 project, where the security proxy raised a Forbidden exception on getattr of __bases__. PyObject_IsInstance(), PyObject_IsSubclass(): After both calls to abstract_get_bases(), where we're setting the TypeError if the return value is NULL, we must first check to see if an exception occurred, and /not/ mask an existing exception. Neil Schemenauer should double check that these changes don't break his ExtensionClass examples (there aren't any test cases for those examples and abstract_get_bases() was added by him in response to problems with ExtensionClass). Neil, please add test cases if possible! I belive this is a bug fix candidate for Python 2.2.2. ---------------------------- --Guido van Rossum (home page: http://www.python.org/~guido/)
This is somewhat off-topic, but I'm hoping maybe someone can give a hint why this only happens on Python 2.2.1.
Ok, here's the story:
I've had a bug report against our pyPgSQL database interface package that retrieving Large Objects doesn't work with Python 2.2.1. The reproducible traceback we get is:
Traceback (most recent call last): File "p.py", line 20, in ? res = cs.fetchone() File "pyPgSQL/PgSQL.py", line 2672, in fetchone return self.__fetchOneRow() File "pyPgSQL/PgSQL.py", line 2281, in __fetchOneRow for _i in range(self.res.nfields): AttributeError: 'str' object has no attribute '__bases__'
This traceback is quite obviously bogus, as self.res.nfields is a Python int and no strings are involved here whatsoever. After some debugging, I found that something very strange happens in a function call that happens in this for loop. Inside the for loop, a function typecast is called, which has this code within:
if isinstance(value, PgBytea) or type(value) is PgLargeObjectType:
This code is causing the problems which result in the bogus traceback later on.
Now in my case, 'value' is of type PgLargeObjectType, which is a custom type from our extension module. PgBytea is a Python class.
Now comes the first very strange observation: Swapping the checks, so that the 'type(value) is PgLargeObjectType' check comes first makes the problem go away. So my conclusion is that there's some problem with isinstance and my custom extension type.
The second strange thing is that this only happens on Python 2.2.1 (Linux, FreeBSD, Windows), but _not_ on Python 2.1.3 or Python 2.3-CVS.
Oh, the problem isn't tied to isinstance(value, PgBytea). Any isinstance check causes it later on.
Of course I'm suspecting that there's some problem with the extension type. Looks like some internal interpreter data gets corrupted. No idea how to debug that, too.
Does anybody have any tips where to look or how to debug this further?
Probably some C code receives an exception and decides to go a different path (rather than propagating the exception), but forgets to call PyErr_Clear(). If you call some other code that raises an exception or calls PyErr_Clear(), the spurious exception is gone; but if you call some other code that *tests* for an exception (usually with PyExc_Occurred() or PyErr_ExceptionMatches()), that code may raise the bogus exception at an unexpected place. So I'd look in your extension for places where it tests for an exception and decides to ignore it but forgets to clear it. It's also possible that this occurs in the Python code (have you tried the 2.2.2 CVS? Use "cvs update -r release22-maint") but if I had to bet, I'd bet on your SQL extension. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
* Guido van Rossum <guido@python.org> [2002-09-27 12:04 -0400]:
It's also possible that this occurs in the Python code (have you tried the 2.2.2 CVS?
Yep, the problem goes away, then.
Use "cvs update -r release22-maint") but if I had to bet, I'd bet on your SQL extension. :-)
How much? ;-) -- Gerhard
participants (4)
-
Gerhard Häring -
Gerhard Häring -
Guido van Rossum -
Michael Hudson