On Mon, Oct 29, 2012 at 7:00 AM, Steve Dower
Richard Oudkerk wrote:
On 28/10/2012 11:52pm, Guido van Rossum wrote:
I'm most interested in feedback on the design of polling.py and scheduling.py, and to a lesser extent on the design of sockets.py; main.py is just an example of how this style works out in practice.
What happens if two tasks try to do a read op (or two tasks try to do a write op) on the same file descriptor? It looks like the second one to do scheduling.block_r(fd) will cause the first task to be forgotten, causing the first task to block forever.
I know I haven't posted my own code yet (coming very soon), but I'd like to put out there that I don't think this is an important sort of question at this time.
Kind of. I think if it was an important use case it might affect the shape of the API. However I can't think of a use case where it might make sense for two tasks to read or write the same file descriptor without some higher-level mediation. (Even at a higher level I find it hard to imagine, except for writing to a common log file -- but even there you want to be sure that individual lines aren't spliced into each other, and the semantics of send() don't prevent that.)
We both have sample schedulers that work well enough to demonstrate the API, but aren't meant to be production ready.
IMO, the important questions are:
- how easy/difficult/flexible/restrictive is it to write a new scheduler as a core Python developer? - how easy/difficult/flexible/restrictive is it to write a new scheduler as an end user? - how easy/difficult/flexible/restrictive is it to write async operations as a core Python developer? - how easy/difficult/flexible/restrictive is it to write async operations as an end user? - how straightforward is it to consume async operations? - how easy is it to write async code that is correct?
Yes, these are all important questions. I'm not sure that end users would be writing new schedulers -- but 3rd party library developers will be, and I suppose that's what you are referring to. My own approach to answering these is to first try to figure out what a typical application would be trying to accomplish. That's why I made a point of implementing a 100% async HTTP client -- it's just quirky enough that it exercises various issues (e.g. switching between line-mode and blob mode, and the need to invoke getaddrinfo()).
Admittedly, I am writing this preemptively knowing that there are a lot of distractions like this in my code (some people are going to be horrified at what I did with file IO :-) Don't worry, it's only for trying the API). Once we know what interface we'll be coding against we can worry about getting the implementation perfect. Also, I imagine we'll find some more volunteers for coding (hopefully people who have done non-blocking stuff in C or similar before) who are currently avoiding the higher-level ideas discussion.
I'm looking forward to it! I suspect we'll be merging our designs shortly... -- --Guido van Rossum (python.org/~guido)