Lightwight socket IO wrapper
James Harris
james.harris.1 at gmail.com
Sun Sep 20 18:36:30 EDT 2015
"Akira Li" <4kir4.1i at gmail.com> wrote in message
news:mailman.37.1442754893.21674.python-list at python.org...
> "James Harris" <james.harris.1 at gmail.com> writes:
>
>> I guess there have been many attempts to make socket IO easier to
>> handle and a good number of those have been in Python.
>>
>> The trouble with trying to improve something which is already well
>> designed (and conciously left as is) is that the so-called
>> improvement
>> can become much more complex and overly elaborate. That can apply to
>> the initial idea, for sure, but when writing helper or convenience
>> functions perhaps it applies more to the temptation to keep adding
>> just a little bit extra. The end result can be overly elaborate such
>> as a framework which is fine where such is needed but is overkill for
>> simpler requirements.
>>
>> Do you guys have any recommendations of some *lightweight* additions
>> to Python socket IO before I write any more of my own? Something
>> built
>> in to Python would be much preferred over any modules which have to
>> be
>> added. I had in the back of my mind that there was a high-level
>> socket-IO library - much as threading was added as a wrapper to the
>> basic thread module - but I cannot find anything above socket. Is
>> there any?
>
> Does ØMQ qualify as lightweight?
It's certainly interesting. It's puzzling, too. For example,
http://zguide.zeromq.org/py:hwserver
The Python code there includes
message = socket.recv()
but given that this is a TCP socket it doesn't look like there is any
way for the stack to know how many bytes to return. Either ZeroMQ layers
another end-to-end protocol on top of TCP (which would be no good) or it
will be guessing (which would not be good either).
There are probably answers to that query but there is a lot of
documentation, including on reliable communication, and that in itself
makes ZeroMQ seem overkill, even if it can be persuaded to do what I
want.
I am impressed that they show code in many languages. I may come back to
it but for the moment it doesn't seem to be what I was looking for. And
it is not built in.
>> A current specific to illustrate where basic socket IO is limited: it
>> normally provides no guarantees over how many bytes are transferred
>> at
>> a time (AFAICS that's true for both streams and datagrams) so the
>> delimiting of messages/records needs to be handled by the sender and
>> receiver. I do already handle some of this myself but I wondered if
>> there was a prebuilt solution that I should be using instead - to
>> save
>> me adding just a little bit extra. ;-)
>
> There are already convenience functions in stdlib such as
> sock.sendall(), sock.sendfile(), socket.create_connection() in
> addition
> to BSD Sockets API.
>
> If you want to extend this list and have specific suggestions; see
> https://docs.python.org/devguide/stdlibchanges.html
That may be a bit overkill just now but it's a good suggestion.
> Or just describe your current specific issue in more detail here.
There are a few things and more crop up as time goes on. For example,
over TCP it would be helpful to have a function to receive a specific
number of bytes or one to read bytes until reaching a certain delimiter
such as newline or zero or space etc. Even better would be to be able to
use the iteration protocol so you could just code next() and get the
next such chunk of read in a for loop. When sending it would be good to
just say to send a bunch of bytes but know that you will get told how
many were sent (or didn't get sent) if it fails. Sock.sendall() doesn't
do that.
I thought UDP would deliver (or drop) a whole datagram but cannot find
anything in the Python documentaiton to guarantee that. In fact
documentation for the send() call says that apps are responsible for
checking that all data has been sent. They may mean that to apply to
stream protocols only but it doesn't state that. (Of course, UDP
datagrams are limited in size so the call may validly indicate
incomplete transmission even when the first part of a big message is
sent successfully.)
Receiving no bytes is taken as indicating the end of the communication.
That's OK for TCP but not for UDP so there should be a way to
distinguish between the end of data and receiving an empty datagram.
The recv calls require a buffer size to be supplied which is a technical
detail. A Python wrapper could save the programmer dealing with that.
Reminder to self: encoding issues.
None of the above is difficult to write and I have written the bits I
need myself but, basically, there are things that would make socket IO
easier and yet still compatible with more long-winded code. So I
wondered if there were already some Python modules which were more
convenient than what I found in the documentation.
James
More information about the Python-list
mailing list