PEP 3156 / Tulip question: write/send callback/future
Hi, I'm looking through PEP 3156 and the Tulip code, and either something is missing or I'm not looking in the right places. I can't find any sort of callback / future return for asynchronous writes, e.g. in transport. Should there be no "data_sent" parallel to "data_received" somewhere? Or, alternatively, "write" returning some sort of future that can be checked later for status? For connections that aren't infinitely fast it's useful to know when the data was actually sent/written, or alternatively if an error has occurred. This is also important for when writing would actually block because of full buffers. boost::asio has such a handler for async_write. Eli
On Fri, Jan 18, 2013 at 6:56 AM, Eli Bendersky <eliben@gmail.com> wrote:
I guess you should read some Twisted tutorial. :-)
The model is a little different. Glyph has convinced me that it works well in practice. We just buffer what is written (when it can't all be sent immediately). This is enough for most apps that don't serve 100MB files. If the buffer becomes too large, the transport will call .pause() on the protocol until it is drained, then it calls .resume(). (The names of these are TBD, maybe they will end up .pause_writing() and .resume_writing().) There are some default behaviors that we can add here too, e.g. suspending the task. -- --Guido van Rossum (python.org/~guido)
On Fri, Jan 18, 2013 at 1:02 PM, Guido van Rossum <guido@python.org> wrote:
Yes, I noticed that Twisted also doesn't have it, so I suspected that influence.
I agree it can be made to work, but how would even simple "done sending" notification work? Or "send error" for that matter? AFAIR, low-level socket async API do provide this information. Are we confident enough it will never be needed to simply hide it away? Eli
On Fri, Jan 18, 2013 at 1:40 PM, Eli Bendersky <eliben@gmail.com> wrote:
AFAIK the Twisted folks have found that most of the time (basically all of the time) you don't need a positive "done sending" notification; when the send eventually *fails*, the transport calls the protocol's connection_lost() method with an exception indicating what failed. -- --Guido van Rossum (python.org/~guido)
Also, ISTR that it's not always possible to consistently have that behavior everywhere (i.e. have it in the first place or fake it where it's not directly available), so it's of somewhat limited utility, since a protocol can't actually rely on it existing. Most behavior that requires it is generally implemented using IPushProducer/IPullProducer (i.e. the pause/resume API Guido mentioned earlier). There's been some attempts at work towards a better producer/consumer API (e.g. supporting things like buffer changes, and generally just simplifying things that seem duplicated amongst transports and consumers/producers) called 'tubes', but I don't think any of that is ready enough to be a template for tulip :) On Fri, Jan 18, 2013 at 11:25 PM, Guido van Rossum <guido@python.org> wrote:
-- cheers lvh
On Fri, Jan 18, 2013 at 6:56 AM, Eli Bendersky <eliben@gmail.com> wrote:
I guess you should read some Twisted tutorial. :-)
The model is a little different. Glyph has convinced me that it works well in practice. We just buffer what is written (when it can't all be sent immediately). This is enough for most apps that don't serve 100MB files. If the buffer becomes too large, the transport will call .pause() on the protocol until it is drained, then it calls .resume(). (The names of these are TBD, maybe they will end up .pause_writing() and .resume_writing().) There are some default behaviors that we can add here too, e.g. suspending the task. -- --Guido van Rossum (python.org/~guido)
On Fri, Jan 18, 2013 at 1:02 PM, Guido van Rossum <guido@python.org> wrote:
Yes, I noticed that Twisted also doesn't have it, so I suspected that influence.
I agree it can be made to work, but how would even simple "done sending" notification work? Or "send error" for that matter? AFAIR, low-level socket async API do provide this information. Are we confident enough it will never be needed to simply hide it away? Eli
On Fri, Jan 18, 2013 at 1:40 PM, Eli Bendersky <eliben@gmail.com> wrote:
AFAIK the Twisted folks have found that most of the time (basically all of the time) you don't need a positive "done sending" notification; when the send eventually *fails*, the transport calls the protocol's connection_lost() method with an exception indicating what failed. -- --Guido van Rossum (python.org/~guido)
Also, ISTR that it's not always possible to consistently have that behavior everywhere (i.e. have it in the first place or fake it where it's not directly available), so it's of somewhat limited utility, since a protocol can't actually rely on it existing. Most behavior that requires it is generally implemented using IPushProducer/IPullProducer (i.e. the pause/resume API Guido mentioned earlier). There's been some attempts at work towards a better producer/consumer API (e.g. supporting things like buffer changes, and generally just simplifying things that seem duplicated amongst transports and consumers/producers) called 'tubes', but I don't think any of that is ready enough to be a template for tulip :) On Fri, Jan 18, 2013 at 11:25 PM, Guido van Rossum <guido@python.org> wrote:
-- cheers lvh
participants (3)
-
Eli Bendersky -
Guido van Rossum -
Laurens Van Houtven