Am 13.03.21 um 01:01 schrieb Ian Haywood:
since with io_uring the queue/ring to append to never blocks for writes, that buffering in twisted would lead to double buffering (as the ring already buffers)
then, io_uring follows a completion IO model (as IOCP). quite different from the "ready to write/read" model (as in epoll etc).
I am wondering, do you already have a design for these aspects of io_uring / twisted? I haven't thought about it in detail, but python presents mmap as bytearray, so I would have thought it fairly easy to write directly to
On 12/03/2021 8:29 pm, Tobias Oberstein wrote: the ring from Protocol.write
fwiw, couple of thoughts: to make it "zero copy" right up into python, a couple of things would be needed (there might be other ways of course ..): the memory to write into on the producer side must be allocated and managed via liburing that memory might for example actually reside inside a NIC and be DMA'ed into main memory ("zero CPU copy") or even directly mapped into the user level address space ("true zero copy") even when taking the very basic case of just transmitting an unstructed bunch of bytes, to make it truly zero copy up into python, sth like https://zero-buffer.readthedocs.io might be needed rather than standard python str/bytes. then, for structured data, if zero-buffer would be used in python lib of flatbuffers for str/bytes https://google.github.io/flatbuffers/ this would result in a truly zero-copy transfer _and_ serialization for native structured objects. all of that with zero/minimal syscalls. in a way, the holy grail;) cheers, /Tobias alternatives for flatbuffers: https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-sbe.html