[Python-Dev] cpython (3.2): bounds check for bad data (thanks amaury)

[Serhiy Storchaka]

On 27.10.12 03:05, philip.jenvey wrote:
> http://hg.python.org/cpython/rev/74d65c746f63
> changeset:   79953:74d65c746f63
> branch:      3.2
> parent:      79941:eb999002916c
> user:        Philip Jenvey <pjenvey at underboss.org>
> date:        Fri Oct 26 17:01:53 2012 -0700
> summary:
>    bounds check for bad data (thanks amaury)

> +        if (strlen(p) > 2 &&


First, it produces compiler warning:

Python/codecs.c: In function ‘PyCodec_SurrogatePassErrors’:
Python/codecs.c:794: warning: pointer targets in passing argument 1 of ‘strlen’ differ in signedness
/usr/include/string.h:397: note: expected ‘const char *’ but argument is of type ‘unsigned char *’

Second, it slowdown the code to 10%:

$ ./python-orig -m timeit -s 'b=b"\xed\xa0\xa0"+b"x"*10000'  'b.decode("utf-8", "surrogatepass")'
100000 loops, best of 3: 12.2 usec per loop
$ ./python -m timeit -s 'b=b"\xed\xa0\xa0"+b"x"*10000'  'b.decode("utf-8", "surrogatepass")'
100000 loops, best of 3: 13.3 usec per loop


I suggest to use the followed code instead:

         if (PyBytes_GET_SIZE(object) - start >= 3 &&