[pypy-issue] Issue #2769: SystemError: unexpected internal exception (please report a bug): <BytecodeCorruption... (pypy/pypy)

Jason Madden issues-reply at bitbucket.org
Fri Mar 2 13:31:36 EST 2018

New issue 2769: SystemError: unexpected internal exception (please report a bug): <BytecodeCorruption...

Jason Madden:

Running PyPy2   on Travis CI, at least two gevent builds in a row failed with this error:
RPython traceback:
    File "pypy_interpreter_1.c", line 34924, in execute_frame
    File "rpython_jit_metainterp.c", line 26150, in ll_portal_runner__Unsigned_Bool_pypy_interpreter
    File "pypy_module_pypyjit.c", line 183, in portal_28
    File "pypy_interpreter_2.c", line 37639, in handle_bytecode__AccessDirect_None
    File "pypy_interpreter_2.c", line 50304, in dispatch_bytecode__AccessDirect_None
    File "pypy_interpreter_2.c", line 54171, in MISSING_OPCODE__AccessDirect_None
  Traceback (most recent call last):
    File "/home/travis/build/gevent/gevent/src/gevent/greenlet.py", line 730, in run
      result = self._run(*self.args, **self.kwargs)
    File "/home/travis/build/gevent/gevent/src/gevent/baseserver.py", line 26, in _handle_and_close_when_done
      return handle(*args_tuple)
    File "/home/travis/build/gevent/gevent/src/gevent/server.py", line 191, in wrap_socket_and_handle
      return self.handle(ssl_socket, address)
    File "/home/travis/build/gevent/gevent/src/gevent/pywsgi.py", line 1506, in handle
    File "/home/travis/build/gevent/gevent/src/gevent/pywsgi.py", line 456, in handle
      result = self.handle_one_request()
    File "/home/travis/build/gevent/gevent/src/gevent/pywsgi.py", line 639, in handle_one_request
      self.requestline = self.read_requestline()
    File "/home/travis/build/gevent/gevent/src/gevent/pywsgi.py", line 587, in read_requestline
      line = self.rfile.readline(MAX_REQUEST_LINE)
    File "/home/travis/.runtimes/versions/pypy5100/lib-python/2.7/socket.py", line 547, in readline
      data = self._sock.recv(self._rbufsize)
    File "/home/travis/build/gevent/gevent/src/gevent/_sslgte279.py", line 458, in recv
      return self.read(buflen)
    File "/home/travis/build/gevent/gevent/src/gevent/_sslgte279.py", line 318, in read
      self._wait(self._read_event, timeout_exc=_SSLErrorReadTimeout)
    File "/home/travis/build/gevent/gevent/src/gevent/_socket2.py", line 196, in _wait
    File "/home/travis/build/gevent/gevent/src/gevent/hub.py", line 615, in wait
      watcher.start(waiter.switch, unique)
    File "/home/travis/build/gevent/gevent/src/gevent/libuv/watcher.py", line 332, in start
    File "/home/travis/build/gevent/gevent/src/gevent/libuv/watcher.py", line 376, in _io_start
      self.start(self._io_callback, pass_events=True)
    File "/home/travis/build/gevent/gevent/src/gevent/_ffi/watcher.py", line 487, in start
      def start(self, callback, *args, **kwargs):
  SystemError: unexpected internal exception (please report a bug): <BytecodeCorruption object at 0x146598288970>; internal traceback was dumped to stderr
  Fri Mar  2 16:36:31 2018 <Greenlet "Greenlet-0" at 0x1465914fb220L: _handle_and_close_when_done(<bound method WSGIServer.wrap_socket_and_handle of, <bound method WSGIServer.do_close of <WSGIServer a, (<socket at 0x14658ff95130L fileno=[Errno 9] Bad f)> failed with SystemError
  EFatal Python error: Aborted
  Stack (most recent call first, approximate line numbers):
    File "/home/travis/build/gevent/gevent/src/gevent/libuv/loop.py", line 389 in run
    File "/home/travis/build/gevent/gevent/src/gevent/hub.py", line 658 in run
    File "/home/travis/.runtimes/versions/pypy5100/lib_pypy/greenlet.py", line 183 in _greenlet_start
    File "<builtin>/?", line 2 in start
(Log https://travis-ci.org/gevent/gevent/jobs/348324982#L838)

Based on the position reported in the log, I took a guess at the test that might be causing the issue and disabled it and everything ran successfully from then on. That test was src/greentest/test__pywsgi.py:TestInputRaw.test_32bit_overflow (which specifically takes PyPy into account). That's the first test in that class to run, and all the other tests following it ran successfully. I don't think that test does any network IO or even switches greenlets, even though the reported error involves both. I don't know if it's possible we were still tearing down the server infrastructure from the last test class?

Here's the code I disabled:
    def test_32bit_overflow(self):
        # https://github.com/gevent/gevent/issues/289
        # Should not raise an OverflowError on Python 2
        data = b'asdf\nghij\n'
        long_data = b'a' * (pywsgi.MAX_REQUEST_LINE + 10)
        long_data += b'\n'
        data = data + long_data
        partial_data = b'qjk\n' # Note terminating \n
        n = 25 * 1000000000
        if hasattr(n, 'bit_length'):
            self.assertEqual(n.bit_length(), 35)
        if not PY3 and not PYPY:
            # Make sure we have the impl we think we do
            self.assertRaises(OverflowError, StringIO(data).readline, n)
        # self.make_input returns a gevent.pywsgi.Input object wrapped around a cStringIO.StringIO object
        i = self.make_input(data, content_length=n)
        # No size hint, but we have too large a content_length to fit
        self.assertEqual(i.readline(), b'asdf\n')
        # Large size hint
        self.assertEqual(i.readline(n), b'ghij\n')
        self.assertEqual(i.readline(n), long_data)

        # Now again with the real content length, assuring we can't read past it
        i = self.make_input(data + partial_data, content_length=len(data) + 1)
        self.assertEqual(i.readline(), b'asdf\n')
        self.assertEqual(i.readline(n), b'ghij\n')
        self.assertEqual(i.readline(n), long_data)
        # Now we've reached content_length so we shouldn't be able to
        # read anymore except the one byte remaining
        self.assertEqual(i.readline(n), b'q')

I couldn't reproduce this locally on macOS or my ubuntu VM, and it popped up suddenly after what should be an unrelated change.

(gevent issue at https://github.com/gevent/gevent/issues/1130; closing in favor of this)

More information about the pypy-issue mailing list