[pypy-dev] cppyy fails to build on gcc 5 and clang

[Antonio Cuni]

Hi Tobias,

On Tue, Jan 17, 2017 at 7:27 PM, Tobias Oberstein <
tobias.oberstein at gmail.com> wrote:

> Hi Antonio,
>
> Fantastic!!
>
> Actually, if this is true (performance), this would be wonderful - I can
> spare my nerves (hello C++, you are a moooonster!), and eat my cake ..
>
> The benchmarks
>
> http://capnpy.readthedocs.io/en/latest/benchmarks.html
>
> for scalar attribute access look awesome (close to mere instance access).
>
> Structured value access (lists/dict): bigger difference. I am also
> concerned about GC pressure - is it still "zero copy"?
>

​yes: under the hood, capnpy objects are represented as immutable strings;
then the generated classes contain accessors which look like this (overly
simplified):

class Point(_Struct):
    @property
    def x​(self):
        offset = statically_known_x_offset + self._offset
        return struct.unpack_from('q', self._buf.s, offset)

where self._buf.s is a string.
The nice thing is that the pypy JIT does a very good job at optimizing
struct.unpack_from: if you look at the generated code, you see that it
loads the 8 bytes directly from the in-memory buffer. So, it's very close
to optimal performance.

For lists is the same: when you look up a list field, you get a wrapper
around the very same buffer, then the custom __getitem__ does the actual
lookup in memory. Note that currently lists on pypy are slowish, but I
should be able to solve the problem in the near future.

The only exception are strings: getting a text attribute means to take a
slice of the buffer. However, if the returned value is short-lived, the JIT
might be able to optimize the slicing away.
(note: there is no automatic conversion to unicode, although I might add an
option for that in the future).

I don't understand what you mean by dicts are there are no dictionaries in
capnproto.

Note also that capnpy is serialization only: there is no support for RPC
stuff.



> Note: I am purely interested in performance on PyPy ..
>
> In general: I thought it would be a good idea to use capnproto C++
> generator, and then cppyy to get the best performance (on pypy). Given
> there is "antocuni/capnpy", do you think this is a pointless endeavour?
>

The original goal of capnpy was to be as fast as possible on PyPy. However,
if you find that C++ + cppyy is faster, I'd be very interested to know :).

ciao,
Anto
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20170118/a8e882e3/attachment.html>