fwiw, I have been trying to get find a conclusive answer to the subquestion: Do Unix domain sockets (either stream or dgram) support flow-control? The nearest I could find is http://stackoverflow.com/questions/1478975/unix-domain-socket-is-there-such-... but the comments seem to contradict each other. seems I'll just try it and see what happens .. /Tobias On 21.10.13 23:33, "David Stainton" <dstainton415@gmail.com> wrote: Hi, I don't know the answer to your question... I've been thinking about flow control lately. I recently wrote a little working prototype VPN in Twisted using IPushProducer and IConsumer interfaces but I have not yet implemented flow control : Here we can see how I proxy between TUN and UDP using producers and consumers: https://github.com/david415/hushVPN/blob/master/hushVPN_service.py Perhaps later on I could add flow control... by having the consumers pause the producers when need be. For now I just churn packets as fast as possible. David On Mon, Oct 21, 2013 at 1:48 PM, Tobias Oberstein <tobias.oberstein@tavendo.de> wrote:

Hi,

with TCP (either remote or loopback) I can have flow-control using the producer-consumer machinery that Twisted provides.

Is that (flow-control / producer-consumer) also available (and practically usable/recommended) with:

a) Unix Domain sockets b) Pipes (spawnProcess)

?

Thanks! /Tobias

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Hi Itamar, once again Twisted surprises me: it just works! I have tested a master/child combo over pipe (both Twisted) with a push producer on the sending leg. It can push around 70MB/s raw binary (16k write size) to the child. Load is evenly on 2 CPU cores each at 80% and stays roughly constant. Memory is flat for both. This is on some years old Macbook (kqueue reactor .. a little faster than select) - tomorrow I repeat on a beefy system/OS. I will do more systematic throughput and also latency measurements .. but this is already not bad at all. Moreso: I really am curious now how that works under the hood, since the pausing/resuming seems to be upper/lower watermark controlled .. it'll resume the producer before the consumer starves. As it should be;) Background: this is part of experiments in preparation for a multi-core capable Autobahn based message broker .. Takeaway: Next time I don't waste time on the internet reading half-baked posts, but just hack away;) /Tobias PS: The following is actually slower than above master/slave pair .. which I also didn't expect: tobias-obersteins-macbook-pro-2:masterchild oberstet$ time dd if=/dev/zero bs=1k count=1000000 | wc -c 1000000+0 records in 1000000+0 records out 1024000000 bytes transferred in 17.454178 secs (58667902 bytes/sec) 1024000000 real 0m17.460s user 0m18.389s sys 0m7.729s On 22.10.13 02:00, "Itamar Turner-Trauring" <itamar@itamarst.org> wrote: On 10/21/2013 04:48 PM, Tobias Oberstein wrote: Flow-control for Pipes / Unix Domains Sockets? Hi, with TCP (either remote or loopback) I can have flow-control using the producer-consumer machinery that Twisted provides. Is that (flow-control / producer-consumer) also available (and practically usable/recommended) with: a) Unix Domain sockets b) Pipes (spawnProcess) I would expect transport.registerProducer to work with both.