[pypy-dev] cppyy fails to build on gcc 5 and clang

[Tobias Oberstein]

>> Holy grail for me (use case being IPC) would be:
>>
>> Python process 1 mmap's a file shared with Python process 2.
>>
>> Python process 1 puts a string into mmap'ed file, pointer to that is
>> "somehow transferred" to process 2 (eg pushing the index into the mmap'ed
>> file over Unix domain socket .. a single uint64), and Python code in
>> process 2 can do stuff with this string _without_ copying - probably via
>> zero_buffer.
>>
>>
> ​Have you actually measured that copying the data between processes is the
> bottleneck? Using shared memory is something I tried also for a client of
> mine but at the end we switched back to pass messages using network because
> the extra complexity was not worth the gain. But again, I suppose it
> depends on the size of the message.

I have measured (and profiled using vmprof) that we have bottlenecks in 
serialization/deserialization .. the overhead is lowest (on PyPy) when 
using a pure CBOR implementation, but still.

Hence my interest in captnproto/flatbuffers.

I also benchmarked Unix domain socket pure bandwidth (in C) .. which is 
very high indeed (UDS only involves a single copy when done in C, not 2 
copies like loopback TCP).

I probably should stop worrying/guessing and try capnpy over UDS and see 
how far this will get us;)

Cheers,
/Tobias


>
> ​ciao,
> Anto​
>