[New-bugs-announce] [issue35589] BaseSelectorEventLoop.sock_sendall() performance regression: extra copy of data

Huazuo Gao report at bugs.python.org
Wed Dec 26 05:04:33 EST 2018

New submission from Huazuo Gao <gaohuazuo at gmail.com>:

Prior to PR 10419, sock_sendall does not make a copy of the data. PR 10419 introduced an extra copy, which may cause problem for code that send a huge chunk of data simultaneously to many peers. Relevant change is:


Bellow is a test that show the regression between 3.7.1 and 3.8-dev


import asyncio
import socket
import os
from subprocess import check_output

loop = asyncio.get_event_loop()

def mem_usage():
    pid = str(os.getpid())
    print(check_output(['ps', '-o', 'rss,comm'], text=True))

async def main():
    data = bytearray(10*10**6)
    data = memoryview(data)
    tasks = []
    for i in range(100):
        s1, s2 = socket.socketpair()
        tasks.append(loop.create_task(loop.sock_sendall(s1, data)))
        tasks.append(loop.create_task(loop.sock_recv(s2, 1)))
    await asyncio.sleep(0.1)
    for t in tasks:
    await asyncio.wait(tasks)




3.7.1: 24724
3.8-dev: 979184

components: asyncio
messages: 332534
nosy: Huazuo Gao, asvetlov, yselivanov
priority: normal
severity: normal
status: open
title: BaseSelectorEventLoop.sock_sendall() performance regression: extra copy of data
type: resource usage
versions: Python 3.8

Python tracker <report at bugs.python.org>

More information about the New-bugs-announce mailing list