[pypy-dev] Performance issues of socket.recv_into()

[hubo]

I have been testing network I/O performance of Python and PyPy. I have used the small script in the attachment to test TCP bandwidth between two powerful servers, each has two 10Gb nics with multi-queue hardware support to form a bond, so the physical bandwidth between the servers are 20Gbps.

The script is a simple socket client/server with multiple connections on multiple threads. The server tries its best to receive all data, and the client tries its best to send data, after 30 sec, the client stops and calculates the data sent.

It seems that there is a huge performance difference for the server to use pypy or CPython:

(python = CPython, pypy = PyPy, 100.102.4.7 is my test server IP address, and 32 is the number of concurrent connections)
With python testio.py -s on another server:

pypy testio.py -c 100.102.4.7 32
total time = 30.05s, total send = 66601 MB, speed = 17.31 Gbps
python testio.py -c 100.102.4.7 32
total time = 30.08s, total send = 67008 MB, speed = 17.40 Gbps


With pypy testio.py -s on another server:

pypy testio.py -c 100.102.4.7 32
total time = 30.36s, total send = 5838 MB, speed = 1.50 Gbps
python testio.py -c 100.102.4.7 32
total time = 30.36s, total send = 5742 MB, speed = 1.48 Gbps

But when I change

        while s.recv_into(recv_buf, 0x200000):

to

        while s.recv(0x200000):

The performance difference disappears, and both PyPy server and CPython server have good performance (17+Gbps)

It is worth to point out that when using socket.recv_into(), PyPy server only uses 100% CPU (one core), while CPython uses much more. Maybe there are some unexpected issues about socket.recv_into, e.g. GIL not released?

P.S. it is interesting that though I thought recv_into() should be more efficient thant recv() since it reduces extra object creation / destruction, the test result shows that recv() outperforms recv_into(), even with CPython. With CPython, it seems server with recv() costs less CPU time than recv_into(), but having the same I/O performance.

2016-08-08


hubo 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20160808/d08e400b/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testio.py
Type: application/octet-stream
Size: 1587 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20160808/d08e400b/attachment.obj>