[New-bugs-announce] [issue41596] Re: Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Mototsugu Emori]

New submission from Mototsugu Emori <mototsugu.emori at jp.ricoh.com>:

When using SSL with websocket, _process_write_backlog function throws IndexError.

---------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 827, in transfer_data
    message = await self.read_message()
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 895, in read_message
    frame = await self.read_data_frame(max_size=self.max_size)
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 971, in read_data_frame
    frame = await self.read_frame(max_size)
  File "/usr/local/lib/python3.7/site-packages/websockets/protocol.py", line 1051, in read_frame
    extensions=self.extensions,
  File "/usr/local/lib/python3.7/site-packages/websockets/framing.py", line 105, in read
    data = await reader(2)
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 679, in readexactly
    await self._wait_for_data('readexactly')
  File "/usr/local/lib/python3.7/asyncio/streams.py", line 473, in _wait_for_data
    await self._waiter
  File "/usr/local/lib/python3.7/asyncio/sslproto.py", line 689, in _process_write_backlog
    del self._write_backlog[0]
IndexError: deque index out of range
---------------------------------------------------------------

Websocket has a keepalive feature and periodically pings the server.
If ping sending process occurs at the same time when sending data, the data operation to the array (_write_backlog) will be duplicated and the error will occur.

---------------------------------------------------------------
asyncio/sslproto.py
---------------------------------------------------------------
class SSLProtocol(protocols.Protocol)
    def _write_appdata(self, data):
        self._write_backlog.append((data, 0)) <--- append data to _write_backlog
        self._write_buffer_size += len(data)
        self._process_write_backlog()

    def _process_write_backlog(self):
        del self._write_backlog[0]  <--- !!!ERROR!!! IndexError: deque index out of range
---------------------------------------------------------------

I made a patch(sslproto.sy.patch) to exclude data operations on the array (_write_backlog).
Could you check it?

Environments:
  debian 4.19.46
  Python 3.7.3
  OpenSSL 1.1.0l  10 Sep 2019

Reference:
  Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range
  https://bugs.python.org/issue37226

----------
components: asyncio
files: sslproto.py.patch
keywords: patch
messages: 375689
nosy: asvetlov, ben.brown, christian.heimes, m_emori, maayank, yselivanov
priority: normal
severity: normal
status: open
title: Re: Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file49409/sslproto.py.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue41596>
_______________________________________