<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<STYLE type=text/css>
BLOCKQUOTE{margin-Top: 0px; margin-Bottom: 0px; margin-Left: 2em}
</STYLE>
<META name=GENERATOR content="MSHTML 11.00.10586.494"><!-- flashmail style begin -->
<STYLE type=text/css>
body {border-width:0;margin:0}
img {border:0;margin:0;padding:0}
</STYLE>
<BASE target=_blank><!-- flashmail style end --></HEAD>
<BODY
style="BORDER-LEFT-WIDTH: 0px; FONT-SIZE: 10.5pt; FONT-FAMILY: 微软雅黑; BORDER-RIGHT-WIDTH: 0px; BORDER-BOTTOM-WIDTH: 0px; COLOR: #000000; MARGIN: 12px; LINE-HEIGHT: 1.5; BORDER-TOP-WIDTH: 0px"
marginheight="0" marginwidth="0">
<DIV>By the way, the result also reproduces on localhost(127.0.0.1), and change
the ctypes.create_string_buffer to bytearray does not change the result.</DIV>
<DIV> </DIV>
<DIV style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; COLOR: #c0c0c0"
align=left>2016-08-08
<HR id=SignNameHR
style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; WIDTH: 122px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px"
align=left>
<SPAN id=_FlashSignName>hubo</SPAN> </DIV>
<HR
style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px">
<BLOCKQUOTE id=ntes-flashmail-quote
style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; PADDING-LEFT: 0px; MARGIN-LEFT: 0px">
<DIV><STRONG>发件人：</STRONG>"hubo" <hubo@jiedaibao.com></DIV>
<DIV><STRONG>发送时间：</STRONG>2016-08-08 22:00</DIV>
<DIV><STRONG>主题：</STRONG>[pypy-dev] Performance issues of
socket.recv_into()</DIV>
<DIV><STRONG>收件人：</STRONG>"PyPy Developer Mailing
List"<pypy-dev@python.org></DIV>
<DIV><STRONG>抄送：</STRONG></DIV>
<DIV> </DIV>
<DIV><STATIONERY>
<DIV>I have been testing network I/O performance of Python and PyPy. I have
used the small script in the attachment to test TCP bandwidth between two
powerful servers, each has two 10Gb nics with multi-queue hardware support to
form a bond, so the physical bandwidth between the servers are 20Gbps.</DIV>
<DIV> </DIV>
<DIV>The script is a simple socket client/server with multiple connections on
multiple threads. The server tries its best to receive all data, and the
client tries its best to send data, after 30 sec, the client stops and
calculates the data sent.</DIV>
<DIV> </DIV>
<DIV>It seems that there is a huge performance difference for the server to
use pypy or CPython:</DIV>
<DIV> </DIV>
<DIV>(python = CPython, pypy = PyPy, 100.102.4.7 is my test server IP address,
and 32 is the number of concurrent connections)</DIV>
<DIV>With python testio.py -s on another server:</DIV>
<DIV> </DIV>
<DIV><STRONG>pypy testio.py -c 100.102.4.7 32<BR></STRONG>total time = 30.05s,
total send = 66601 MB, speed = 17.31 Gbps<BR><STRONG>python testio.py -c
100.102.4.7 32<BR></STRONG>total time = 30.08s, total send = 67008 MB, speed =
17.40 Gbps<BR></DIV>
<DIV><STRONG></STRONG> </DIV>
<DIV>
<DIV>With pypy testio.py -s on another server:</DIV></DIV>
<DIV> </DIV>
<DIV><STRONG>pypy testio.py -c 100.102.4.7 32<BR></STRONG>total time = 30.36s,
total send = 5838 MB, speed = 1.50 Gbps<BR><STRONG>python testio.py -c
100.102.4.7 32<BR></STRONG>total time = 30.36s, total send = 5742 MB, speed =
1.48 Gbps</DIV>
<DIV> </DIV>
<DIV>But when I change</DIV>
<DIV> </DIV>
<DIV> while s.recv_into(recv_buf,
0x200000):<BR></DIV>
<DIV>to</DIV>
<DIV> </DIV>
<DIV> while
s.recv(0x200000):<BR></DIV>
<DIV>The performance difference disappears, and both PyPy server and CPython
server have good performance (17+Gbps)</DIV>
<DIV> </DIV>
<DIV>It is worth to point out that when using socket.recv_into(), PyPy server
only uses 100% CPU (one core), while CPython uses much more. Maybe
there are some unexpected issues about socket.recv_into, e.g. GIL not
released?</DIV>
<DIV> </DIV>
<DIV>P.S. it is interesting that though I thought recv_into() should be more
efficient thant recv() since it reduces extra object creation / destruction,
the test result shows that recv() outperforms recv_into(), even with CPython.
With CPython, it seems server with recv() costs less CPU time than
recv_into(), but having the same I/O performance.</DIV>
<DIV> </DIV>
<DIV style="FONT-SIZE: 10pt; FONT-FAMILY: Verdana; COLOR: #c0c0c0">
<DIV align=left>2016-08-08</DIV>
<HR id=SignNameHR
style="BORDER-TOP: #c0c0c0 1px solid; HEIGHT: 1px; BORDER-RIGHT: 0px; WIDTH: 122px; BORDER-BOTTOM: 0px; BORDER-LEFT: 0px"
align=left>
<SPAN id=_FlashSignName>hubo</SPAN>
</DIV></STATIONERY></DIV></BLOCKQUOTE></BODY></HTML>