[pypy-dev] Debugging "cannot find gc roots!"

Maciej Fijalkowski fijall at gmail.com
Tue Jun 28 09:36:08 CEST 2011


On Tue, Jun 28, 2011 at 2:09 AM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>
>
> On Mon, Jun 27, 2011 at 3:45 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>
>>
>> On Sat, Jun 25, 2011 at 1:11 AM, Armin Rigo <arigo at tunes.org> wrote:
>>>
>>> Hi Alex,
>>>
>>> On Sat, Jun 25, 2011 at 3:11 AM, Alex Gaynor <alex.gaynor at gmail.com>
>>> wrote:
>>> > In running my application I hit a "cannot find gc roots!", it
>>> > reproduces
>>> > reliably, but it's a pain to isolate because it occurs in a very large
>>> > application, running under paste, with threads :(
>>>
>>> You should rebuild pypy with "make lldebug".  Then gdb should stop as
>>> soon as the crash is found.
>>>
>>>
>>> Armin
>>
>> Hrm, this is pretty strange, running an lldebug build, not under gdb
>> reliably produces a crash:
>>
>> alex at devalex:~/ans/venv/answeb$ ANS_INSTANCE=dwc pypy
>> ../lib/pypy/bin/paster serve --reload alex-dev.ini
>> Starting subprocess with file monitor
>> overriding ans instance based on environment: dwc
>> Starting server in PID 6809.
>> serving on 0.0.0.0:5050 view at http://127.0.0.1:5050
>> cannot find gc roots!
>> debug_alloc.h: 9 mallocs left (use PYPY_ALLOC=1 to see the list)
>>
>> However, running under gdb results in broken pip errors from a socket
>> (this is a web app in case that wasn't clear):
>>
>> alex at devalex:~/ans/venv/answeb$ ANS_INSTANCE=dwc gdb --args pypy
>> ../lib/pypy/bin/paster serve --reload alex-dev.ini
>> GNU gdb (GDB) 7.1-ubuntu
>> Copyright (C) 2010 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-linux-gnu".
>> For bug reporting instructions, please see:
>> <http://www.gnu.org/software/gdb/bugs/>...
>> Reading symbols from /home/alex/ans/venv/bin/pypy...(no debugging symbols
>> found)...done.
>> (gdb) run
>> Starting program: /home/alex/ans/venv/bin/pypy ../lib/pypy/bin/paster
>> serve --reload alex-dev.ini
>> warning: the debug information found in "/lib/ld-2.11.1.so" does not match
>> "/lib/ld-linux.so.2" (CRC mismatch).
>>
>> warning: the debug information found in "/usr/lib/debug//lib/ld-2.11.1.so"
>> does not match "/lib/ld-linux.so.2" (CRC mismatch).
>>
>> warning: the debug information found in "/usr/lib/debug/lib/ld-2.11.1.so"
>> does not match "/lib/ld-linux.so.2" (CRC mismatch).
>>
>> [Thread debugging using libthread_db enabled]
>> Starting subprocess with file monitor
>> overriding ans instance based on environment: dwc
>> Starting server in PID 6743.
>> serving on 0.0.0.0:5050 view at http://127.0.0.1:5050
>> ----------------------------------------
>> Exception happened during processing of request from ('10.160.27.96',
>> 44587)
>> Traceback (most recent call last):
>>   File "/home/alex/ans/venv/lib/pypy/site-packages/paste/httpserver.py",
>> line 1068, in process_request_in_thread
>>     self.finish_request(request, client_address)
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 323, in finish_request
>>     self.RequestHandlerClass(request, client_address, self)
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 641, in __init__
>>     self.finish()
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 694, in finish
>>     self.wfile.flush()
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/modified-2.7/socket.py",
>> line 348, in flush
>>     self._sock.sendall(view[write_offset:write_offset+buffer_size])
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/modified-2.7/socket.py",
>> line 266, in sendall
>>     self._sock.sendall(data, flags)
>> error: [Errno 32] Broken pipe
>> ----------------------------------------
>> ----------------------------------------
>> Exception happened during processing of request from ('10.160.27.96',
>> 44608)
>> Traceback (most recent call last):
>>   File "/home/alex/ans/venv/lib/pypy/site-packages/paste/httpserver.py",
>> line 1068, in process_request_in_thread
>>     self.finish_request(request, client_address)
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 323, in finish_request
>>     self.RequestHandlerClass(request, client_address, self)
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 641, in __init__
>>     self.finish()
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/2.7/SocketServer.py", line
>> 694, in finish
>>     self.wfile.flush()
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/modified-2.7/socket.py",
>> line 348, in flush
>>     self._sock.sendall(view[write_offset:write_offset+buffer_size])
>>   File "/home/alex/ans/venv/lib/pypy/lib-python/modified-2.7/socket.py",
>> line 266, in sendall
>>     self._sock.sendall(data, flags)
>> error: [Errno 32] Broken pipe
>> ----------------------------------------
>> ----------------------------------------
>>
>>
>> The only thing worth noting is that this a 32-bit PyPy on a 64-bit OS (the
>> gdb is also 64-bits, so if that matters the gdb runs may not be valid).  I'm
>> not sure whether I see the socket issue with gdb because its dieing earlier
>> than the gc roots issue, or if it's because the error doesn't appear for
>> some reason, and therefore it gets farther and hits a second issue.
>>
>> Alex
>>
>>
>> --
>> "I disapprove of what you say, but I will defend to the death your right
>> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>> "The people's good is the highest law." -- Cicero
>>
>
> FWIW under shadowstack I'm seeing the broken pipe error as well, so I guess
> gdb is just making the error not happen.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>
>

You have to make gdb ignore SIGPIPE signals (pass it to application),
then stuff can work normally. You can also just do ulimit -c <some
large no> and then do gdb pypy-c core


More information about the pypy-dev mailing list