[pypy-dev] ctypes - PyPy 1.8 slower than Python 2.6.5

[Sébastien Volle]

Hi all,

My team is working on a project of fast packet sniffers and I'm comparing
performance between different languages.
So, we came up with a simple ARP sniffer that I ported to Python using
ctypes.

During my investigations, I turned out that using ctypes, PyPy 1.8 is
4x slower than CPython 2.6.5.
After looking at the PyPy buglist, it's seems there are couple open issues
about ctypes so I figured I would ask you guys first before filing a new
bug.

I'm pretty new to ctypes and pypy so I'm not sure I understand what's
going. My program seems to spend a lot of time in
ctypes/function.py:_convert_args though, has the following profile trace
demonstrates:

$ pypy -m cProfile -s time arp.py
Packet buffer now at 0x9CCECB2
Capture started
elapsed time  : 3983.84ms
Total packets : 35571
packets/s     : 8928.81

         7839198 function calls (7838340 primitive calls) in 4.105 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    69876    0.546    0.000    1.584    0.000 function.py:480(_convert_args)
   214696    0.256    0.000    0.429    0.000 structure.py:236(_subarray)
  1052195    0.206    0.000    0.206    0.000 {isinstance}
   632437    0.192    0.000    0.192    0.000 {method 'append' of 'list'
objects}
   175326    0.187    0.000    0.407    0.000 function.py:350(_call_funcptr)
        1    0.173    0.173    4.105    4.105 arp.py:1(<module>)
   209628    0.158    0.000    0.587    0.000 primitive.py:272(from_param)
   214696    0.149    0.000    0.963    0.000 structure.py:90(__get__)
    71144    0.143    0.000    0.198    0.000 structure.py:216(__new__)
   106713    0.130    0.000    0.208    0.000 array.py:70(_CData_output)
   105450    0.124    0.000    2.281    0.000 function.py:689(__call__)
    69876    0.123    0.000    1.943    0.000 function.py:278(__call__)
   321412    0.102    0.000    0.102    0.000 {method 'fromaddress' of
'Array' objects}
   209628    0.088    0.000    0.811    0.000 function.py:437(_conv_param)
   179125    0.083    0.000    0.083    0.000 {method 'fieldaddress' of
'StructureInstance' objects}
    69883    0.080    0.000    0.122    0.000 primitive.py:308(__init__)
    71142    0.076    0.000    0.320    0.000 structure.py:174(from_address)
   105450    0.075    0.000    0.145    0.000 function.py:593(_build_result)
   139755    0.072    0.000    0.120    0.000
primitive.py:64(generic_xxx_p_from_param)
   107983    0.070    0.000    0.125    0.000 basics.py:60(_CData_output)
   209828    0.062    0.000    0.062    0.000 {method 'get' of 'dict'
objects}
   107986    0.055    0.000    0.055    0.000 {method '__new__' of
'_ctypes.primitive.SimpleType' objects}
    71142    0.052    0.000    0.372    0.000 pointer.py:77(getcontents)
    35578    0.052    0.000    0.125    0.000 pointer.py:62(__init__)
    35576    0.050    0.000    0.062    0.000 pointer.py:83(setcontents)
   106713    0.047    0.000    0.047    0.000 {method '__new__' of
'_ctypes.array.ArrayMeta' objects}
   139750    0.043    0.000    0.181    0.000
primitive.py:84(from_param_void_p)
   209625    0.043    0.000    0.691    0.000 basics.py:50(get_ffi_param)
283592/283435    0.041    0.000    0.041    0.000 {len}
    71144    0.040    0.000    0.040    0.000 {method '__new__' of
'_ctypes.structure.StructOrUnionMeta' objects}
   105450    0.039    0.000    0.039    0.000 {method 'free_temp_buffers'
of '_ffi.FuncPtr' objects}
   176683    0.037    0.000    0.037    0.000 {hasattr}
    35571    0.037    0.000    0.423    0.000 pcap.py:89(next)


Anyway, I attached my full project to this mail, including a sample pcap
capture file. It requires libpcap to be installed on your system. Run
arp.py to run the sniffer on the included demo.pcap capture file.
I realize I should try and narrow down the issue some more, but I can't
really afford to spend too much time on this right now so hopefully, this
will be a least a bit helpful.

Thanks!

Regards,
Sébastien
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120213/d88bc38f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctypes-test.tar.bz2
Type: application/x-bzip2
Size: 242538 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120213/d88bc38f/attachment-0001.bin>