Hi Tobias, On Tue, Jan 17, 2017 at 7:27 PM, Tobias Oberstein < tobias.oberstein@gmail.com> wrote:
Hi Antonio,
Fantastic!!
Actually, if this is true (performance), this would be wonderful - I can spare my nerves (hello C++, you are a moooonster!), and eat my cake ..
The benchmarks
http://capnpy.readthedocs.io/en/latest/benchmarks.html
for scalar attribute access look awesome (close to mere instance access).
Structured value access (lists/dict): bigger difference. I am also concerned about GC pressure - is it still "zero copy"?
yes: under the hood, capnpy objects are represented as immutable strings; then the generated classes contain accessors which look like this (overly simplified): class Point(_Struct): @property def x(self): offset = statically_known_x_offset + self._offset return struct.unpack_from('q', self._buf.s, offset) where self._buf.s is a string. The nice thing is that the pypy JIT does a very good job at optimizing struct.unpack_from: if you look at the generated code, you see that it loads the 8 bytes directly from the in-memory buffer. So, it's very close to optimal performance. For lists is the same: when you look up a list field, you get a wrapper around the very same buffer, then the custom __getitem__ does the actual lookup in memory. Note that currently lists on pypy are slowish, but I should be able to solve the problem in the near future. The only exception are strings: getting a text attribute means to take a slice of the buffer. However, if the returned value is short-lived, the JIT might be able to optimize the slicing away. (note: there is no automatic conversion to unicode, although I might add an option for that in the future). I don't understand what you mean by dicts are there are no dictionaries in capnproto. Note also that capnpy is serialization only: there is no support for RPC stuff.
Note: I am purely interested in performance on PyPy ..
In general: I thought it would be a good idea to use capnproto C++ generator, and then cppyy to get the best performance (on pypy). Given there is "antocuni/capnpy", do you think this is a pointless endeavour?
The original goal of capnpy was to be as fast as possible on PyPy. However, if you find that C++ + cppyy is faster, I'd be very interested to know :). ciao, Anto