Lightwight socket IO wrapper
cs at zip.com.au
Mon Sep 21 08:27:13 CEST 2015
On 21Sep2015 12:40, Chris Angelico <rosuav at gmail.com> wrote:
>On Mon, Sep 21, 2015 at 11:55 AM, Cameron Simpson <cs at zip.com.au> wrote:
>> On 21Sep2015 10:34, Chris Angelico <rosuav at gmail.com> wrote:
>>> If you're going to add sequencing and acknowledgements to UDP,
>>> wouldn't it be easier to use TCP and simply prefix every message with
>>> a two-byte length?
>> Frankly, often yes. That's what I do. (different length encoding, but
>Out of interest, what encoding?
NB: this is for binary protocols.
I don't like embedding arbitrary size limits in protocols or data formats if I
can easily avoid it. So (for my home grown binary protocols) I encode unsigned
integers as big endian octets with the top bit meaning "another octet follows"
and the bottom 7 bits going to the value. So my packets look like:
For sizes below 128, one byte of length. For sizes 128-16383, two bytes. And so
on. Compact yet unbounded.
My new protocols ar probably going to derive from the scheme implemented in
the code cited below. "New" means as of some weeks ago, when I completely
rewrote a painful ad hoc protocol of mine and pulled out the general features
into what follows.
The actual packet format is implemented by the Packet class at the bottom of
Simple and flexible.
As for using that data format multiplexed with multiple channels, see the
PacketConnection class here:
Broadly, the packets are length[tag,flags[,channel#],payload] and one
implements whatever semantics one needs on top of that.
You can see this exercised over UNIX pipes and TCP streams in the unit tests
On the subject of packet stuffing, my preferred loop for that is visible in the
PacketConnection._send worker thread method, which goes:
fp = self._send_fp
Q = self._sendQ
for P in Q:
sig = (P.channel, P.tag, P.is_request)
if sig in self.__sent:
raise RuntimeError("second send of %s" % (P,))
In short: get packets from the queue and write them to the stream buffer. If
the queue gets empty, _only then_ flush the buffer. This assures synchronicity
in comms while giving the IO library a chance to fill a buffer with several
Cameron Simpson <cs at zip.com.au>
ERROR 155 - You can't do that. - Data General S200 Fortran error code list
More information about the Python-list