[New-bugs-announce] [issue26799] gdb support fails with "Invalid cast."

[Thomas]

New submission from Thomas:

Trying to use any kind of python gdb integration results in the following error:

(gdb) py-bt
Traceback (most recent call first):
Python Exception <class 'gdb.error'> Invalid cast.: 
Error occurred in Python command: Invalid cast.

I have tracked it down to the _type_... globals, and I am able to fix it with the following commands:

(gdb) pi
>>> # Look up the gdb.Type for some standard types:
... _type_char_ptr = gdb.lookup_type('char').pointer() # char*
>>> _type_unsigned_char_ptr = gdb.lookup_type('unsigned char').pointer() # unsigned char*
>>> _type_void_ptr = gdb.lookup_type('void').pointer() # void*
>>> _type_unsigned_short_ptr = gdb.lookup_type('unsigned short').pointer()
>>> _type_unsigned_int_ptr = gdb.lookup_type('unsigned int').pointer()

After this, it works correctly. I was able to workaround it by making a fix_globals that resets the globals on each gdb.Command. I do not understand why the originally initialized types are not working properly. It feels like gdb-inception trying to debug python within a gdb that debugs cpython while executing python code.

I have tried this using hg/default cpython (--with-pydebug --without-pymalloc --with-valgrind --enable-shared) 
1) System install of gdb 7.11, linked against system libpython 3.5.1.
2) Custom install of gdb 7.11.50.20160411-git, the debug cpython I am trying to debug

----------
components: Demos and Tools
messages: 263690
nosy: tilsche
priority: normal
severity: normal
status: open
title: gdb support fails with "Invalid cast."
type: crash
versions: Python 3.6

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26799>
_______________________________________