Hi all,<div><br></div><div>My team is working on a project of fast packet sniffers and I'm comparing performance between different languages.</div><div>So, we came up with a simple ARP sniffer that I ported to Python using ctypes.</div>
<div><br></div><div>During my investigations, I turned out that using ctypes, PyPy 1.8 is 4x slower than CPython 2.6.5.</div><div>After looking at the PyPy buglist, it's seems there are couple open issues about ctypes so I figured I would ask you guys first before filing a new bug.</div>
<div><br></div><div>I'm pretty new to ctypes and pypy so I'm not sure I understand what's going. My program seems to spend a lot of time in ctypes/function.py:_convert_args though, has the following profile trace demonstrates:</div>
<div><br></div><div><div>$ pypy -m cProfile -s time arp.py</div><div>Packet buffer now at 0x9CCECB2</div><div>Capture started</div><div>elapsed time : 3983.84ms</div><div>Total packets : 35571</div><div>packets/s : 8928.81</div>
<div><br></div><div> 7839198 function calls (7838340 primitive calls) in 4.105 seconds</div><div><br></div><div> Ordered by: internal time</div><div><br></div><div> ncalls tottime percall cumtime percall filename:lineno(function)</div>
<div> 69876 0.546 0.000 1.584 0.000 function.py:480(_convert_args)</div><div> 214696 0.256 0.000 0.429 0.000 structure.py:236(_subarray)</div><div> 1052195 0.206 0.000 0.206 0.000 {isinstance}</div>
<div> 632437 0.192 0.000 0.192 0.000 {method 'append' of 'list' objects}</div><div> 175326 0.187 0.000 0.407 0.000 function.py:350(_call_funcptr)</div><div> 1 0.173 0.173 4.105 4.105 arp.py:1(<module>)</div>
<div> 209628 0.158 0.000 0.587 0.000 primitive.py:272(from_param)</div><div> 214696 0.149 0.000 0.963 0.000 structure.py:90(__get__)</div><div> 71144 0.143 0.000 0.198 0.000 structure.py:216(__new__)</div>
<div> 106713 0.130 0.000 0.208 0.000 array.py:70(_CData_output)</div><div> 105450 0.124 0.000 2.281 0.000 function.py:689(__call__)</div><div> 69876 0.123 0.000 1.943 0.000 function.py:278(__call__)</div>
<div> 321412 0.102 0.000 0.102 0.000 {method 'fromaddress' of 'Array' objects}</div><div> 209628 0.088 0.000 0.811 0.000 function.py:437(_conv_param)</div><div> 179125 0.083 0.000 0.083 0.000 {method 'fieldaddress' of 'StructureInstance' objects}</div>
<div> 69883 0.080 0.000 0.122 0.000 primitive.py:308(__init__)</div><div> 71142 0.076 0.000 0.320 0.000 structure.py:174(from_address)</div><div> 105450 0.075 0.000 0.145 0.000 function.py:593(_build_result)</div>
<div> 139755 0.072 0.000 0.120 0.000 primitive.py:64(generic_xxx_p_from_param)</div><div> 107983 0.070 0.000 0.125 0.000 basics.py:60(_CData_output)</div><div> 209828 0.062 0.000 0.062 0.000 {method 'get' of 'dict' objects}</div>
<div> 107986 0.055 0.000 0.055 0.000 {method '__new__' of '_ctypes.primitive.SimpleType' objects}</div><div> 71142 0.052 0.000 0.372 0.000 pointer.py:77(getcontents)</div><div>
35578 0.052 0.000 0.125 0.000 pointer.py:62(__init__)</div><div> 35576 0.050 0.000 0.062 0.000 pointer.py:83(setcontents)</div><div> 106713 0.047 0.000 0.047 0.000 {method '__new__' of '_ctypes.array.ArrayMeta' objects}</div>
<div> 139750 0.043 0.000 0.181 0.000 primitive.py:84(from_param_void_p)</div><div> 209625 0.043 0.000 0.691 0.000 basics.py:50(get_ffi_param)</div><div>283592/283435 0.041 0.000 0.041 0.000 {len}</div>
<div> 71144 0.040 0.000 0.040 0.000 {method '__new__' of '_ctypes.structure.StructOrUnionMeta' objects}</div><div> 105450 0.039 0.000 0.039 0.000 {method 'free_temp_buffers' of '_ffi.FuncPtr' objects}</div>
<div> 176683 0.037 0.000 0.037 0.000 {hasattr}</div><div> 35571 0.037 0.000 0.423 0.000 pcap.py:89(next)</div></div><div><br></div><div><br></div><div>Anyway, I attached my full project to this mail, including a sample pcap capture file. It requires libpcap to be installed on your system. Run arp.py to run the sniffer on the included demo.pcap capture file.</div>
<div>I realize I should try and narrow down the issue some more, but I can't really afford to spend too much time on this right now so hopefully, this will be a least a bit helpful.</div><div><br></div><div>Thanks!</div>
<div><br></div><div>Regards,</div><div>Sébastien</div>