socket.sendall() "counter" parameter
With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally: sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent) This would both allow to not lose information on error and avoid keeping track of the total data being sent, which usually requires and extra len() call. E.g. when sending a file: file = open('somefile', 'rb') total = 0 while True: chunk = file.read(8192) if not chunk: break sock.sendall(chunk, counter=total) Thoughts? -- Giampaolo - http://grodola.blogspot.com
On Fri, Apr 18, 2014 at 9:34 PM, Giampaolo Rodola'
With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally:
sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent)
That doesn't work with a straight integer; there's no way for sendall() to modify your integer. It'd need to be some mutable type, or else the new value would have to be returned somewhere. ChrisA
On Fri, Apr 18, 2014 at 1:52 PM, Chris Angelico
On Fri, Apr 18, 2014 at 9:34 PM, Giampaolo Rodola'
wrote: With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally:
sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent)
That doesn't work with a straight integer; there's no way for sendall() to modify your integer. It'd need to be some mutable type, or else the new value would have to be returned somewhere.
Ouch! It seems you're right, I didn't take that into account. -- Giampaolo - http://grodola.blogspot.com
On 2014-04-18, at 13:52 , Chris Angelico
On Fri, Apr 18, 2014 at 9:34 PM, Giampaolo Rodola'
wrote: With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally:
sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent)
That doesn't work with a straight integer; there's no way for sendall() to modify your integer. It'd need to be some mutable type, or else the new value would have to be returned somewhere.
sendall could just raise a subtype of socket.error with the actually-sent-bytes count attached couldn't it?
On Fri, 18 Apr 2014 13:34:45 +0200
"Giampaolo Rodola'"
With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally:
sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent)
This would both allow to not lose information on error and avoid keeping track of the total data being sent, which usually requires and extra len() call. E.g. when sending a file:
file = open('somefile', 'rb') total = 0 while True: chunk = file.read(8192) if not chunk: break sock.sendall(chunk, counter=total)
Why not simply use send() in such cases? (or, if you want something file-like, call makefile() and then write()) Regards Antoine.
On Fri, Apr 18, 2014 at 2:04 PM, Antoine Pitrou
On Fri, 18 Apr 2014 13:34:45 +0200 "Giampaolo Rodola'"
wrote: With current socket.sendall() implementation if an error occurs it's impossible to tell how much data was sent. As such I'm wondering whether it would make sense to add a "counter" parameter which gets incremented internally:
sent = 0 try: sock.sendall(data, counter=sent) except socket.error as err: priint("only %s bytes were sent" % sent)
This would both allow to not lose information on error and avoid keeping track of the total data being sent, which usually requires and extra len() call. E.g. when sending a file:
file = open('somefile', 'rb') total = 0 while True: chunk = file.read(8192) if not chunk: break sock.sendall(chunk, counter=total)
Why not simply use send() in such cases? (or, if you want something file-like, call makefile() and then write())
Regards
Antoine.
send() requires to keep track of how much data has actually been sent, hence you need to add some additional logic to resend the missing (tail) part (e.g.: http://hg.python.org/cpython/file/7433f7bce880/Lib/asyncio/unix_events.py#l3... ). That is is fine, but also a bit annoying when you're lazy and grumpy like me. ;-) Actually what would really be useful for sendall() in order to be as "nicer" as possible in terms of usability would be to: 1 - return the number of bytes sent instead of None; that would spare you from using "total += len(data)" on every iteration. 2 - in case of error set a "sent" attribute to the returned (socket.error) exception. I'm not sure how #2 is feasible in practice though nor if it's acceptable to have only certain socket.error exceptions providing extra attributes. -- Giampaolo - http://grodola.blogspot.com
On Fri, 18 Apr 2014 14:28:22 +0200
"Giampaolo Rodola'"
2 - in case of error set a "sent" attribute to the returned (socket.error) exception.
If you want to write the logic to re-send the remaining data on error, why not also write the logic to use send() and re-send the remaining data on success? AFAICT, it's the same logic. Regards Antoine.
On Fri, Apr 18, 2014 at 2:35 PM, Antoine Pitrou
On Fri, 18 Apr 2014 14:28:22 +0200 "Giampaolo Rodola'"
wrote: 2 - in case of error set a "sent" attribute to the returned
(socket.error)
exception.
If you want to write the logic to re-send the remaining data on error, why not also write the logic to use send() and re-send the remaining data on success? AFAICT, it's the same logic.
Regards
Antoine.
Tipically in case of error on sendall() you don't want to resend anything because most of the times it's a disconnection error (and never, say, a "retry" error such as EAGAIN). The use case I'm thinking about, and which would probably be the most common one, is to figure out how much data was sent (via exc.sent or something), add it to "total", reconnect and resume the transfer from where you left. -- Giampaolo - http://grodola.blogspot.com
On 2014-04-18, at 14:35 , Antoine Pitrou
On Fri, 18 Apr 2014 14:28:22 +0200 "Giampaolo Rodola'"
wrote: 2 - in case of error set a "sent" attribute to the returned (socket.error) exception.
If you want to write the logic to re-send the remaining data on error, why not also write the logic to use send() and re-send the remaining data on success? AFAICT, it's the same logic.
sendall keeps trying to send stuff until finished or there's an error, so surely there's no point in wrapping *that* in yet an other send loop? But the amount of data which was successfully sent can be useful for reporting, logging, or to check whether complete segments were sent (if the protocol provides such a thing).
Never gonna happen, for all the reasons brought up. Give it up. :-) -- --Guido van Rossum (python.org/~guido)
On Fri, Apr 18, 2014 at 4:40 PM, Guido van Rossum
Never gonna happen, for all the reasons brought up. Give it up. :-)
-- --Guido van Rossum (python.org/~guido)
Fair enough (I was not convinced either). =) -- Giampaolo - http://grodola.blogspot.com
participants (5)
-
Antoine Pitrou
-
Chris Angelico
-
Giampaolo Rodola'
-
Guido van Rossum
-
Masklinn