[Python-Dev] memcmp performance

[Richard Saunders]

Hey,

> I have been doing some performance experiments with memcmp, and I was
> surprised that memcmp wasn't faster than it was in Python.  I did a whole, 
> long analysis and came up with some very simple results.

Paul Svensson suggested I post as much as I can as text, as people would be more likely to read it.
So, here's the basic ideas:

(1) memcmp is surprisingly slow on some Intel gcc platforms (Linux)
        On several Linux, Intel platforms, memcmp was 2-3x slower than 
        a simple, portable C function (with some optimizations)

(2) The problem: If you compile C programs with gcc with any optimization on, 
     it will replace all memcmp calls with an assembly language stub: rep cmpsb
     instead of the memcmp call.

(3) rep cmpsb seems like it would be faster, but it really isn't: 
      this completely bypasses the memcmp.S, memcmp_sse3.S
      and memcmp_sse4.S in glibc which are typically  faster.

(4) The basic conclusion is that the Python baseline on 
     Intel gcc platforms should probably be compiled with -fno-builtin-memcmp
     so we "avoid" gcc's  memcmp optimization.

The numbers are all in the paper: I will endeavor to try to generate a text form
of all the tables so it's easier to read.  This is much first in the Python dev
arena, so I went a little overboard with my paper below. ;)

  Gooday,

  Richie

> Before I put in a tracker bug report, I wanted to present my findings
> and make sure they were repeatable to others (isn't that the nature
> of science? ;)   as well as offer discussion.
>
> The analysis is a pdf and is here: 
>     http://www.picklingtools.com/study.pdf
> The testcases are a tarball here:
>    http://www.picklingtools.com/PickTest5.tar.gz
>
> I have three basic recommendations in the study: I am
> curious what other people think.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20111020/c910868a/attachment.html>