On Sun, Oct 7, 2012 at 9:44 PM, Ben Darnell wrote:

On Sun, Oct 7, 2012 at 7:01 PM, Guido van Rossum wrote:

...

As long as it's not so low-level that other people shy away from it.

That depends on the target audience. The low-level IOLoop and Reactor are pretty similar -- you can implement one in terms of the other -- but as you move up the stack cross-compatibility becomes harder. For example, if I wanted to implement tornado's IOStreams in twisted, I wouldn't start with the analogous class in twisted (Protocol?), I'd go down to the Reactor and build from there, so putting something IOStream or Protocol in asycore2 wouldn't do much to unify the two worlds. (it would help people build async stuff with the stdlib alone, but at that point it becomes more like a peer or competitor to tornado and twisted instead of a bridge between them)

Sure. And of course we can't expect Twisted and Tornado to just merge projects. They each have different strengths and weaknesses and they each have strong opinions on how things should be done. I do get your point that none of that is incompatible with a shared reactor specification.

...

I also have a feeling that one way or another this will require cooperation between the Twisted and Tornado developers in order to come up with a compromise that both are willing to conform to in a meaningful way. (Unfortunately I don't know how to define "meaningful way" more precisely here. I guess the idea is that almost all things *using* an event loop use the standardized abstract API without caring whether underneath it's Tornado, Twisted, or some simpler thing in the stdlib.

I'd phrase the goal as being able to run both Tornado and Twisted in the same thread without any piece of code needing to know about both systems. I think that's achievable as far as core functionality goes. I expect both sides have some lesser-used functionality that might not make it into the stdlib version, but as long as it's possible to plug in a "real" IOLoop or Reactor when needed it should be OK.

Sounds good. I think a reactor is always going to be an extension of the shared spec. [...]

...

That's interesting. I haven't found the need for this yet. Is it really so common that you can't write this as a Future() constructor plus a call to add_done_callback()? Or is there some subtle semantic difference?

It's a Future constructor, a (conditional) add_done_callback, plus the calls to set_result or set_exception and the with statement for error handling. In full:

def future_wrap(f): @functools.wraps(f) def wrapper(*args, **kwargs): future = Future() if kwargs.get('callback') is not None: future.add_done_callback(kwargs.pop('callback')) kwargs['callback'] = future.set_result def handle_error(typ, value, tb): future.set_exception(value) return True with ExceptionStackContext(handle_error): f(*args, **kwargs) return future return wrapper

Hmm... I *think* it automatically adds a special keyword 'callback' to the *call* site so that you can do things like fut = some_wrapped_func(blah, callback=my_callback) and then instead of using yield to wait for the callback, put the continuation of your code in the my_callback() function. But it also seems like it passes callback=future.set_result as the callback to the wrapped function, which looks to me like that function was apparently written before Futures were widely used. This seems pretty impure to me and I'd like to propose a "future" where such functions either be given the Future where the result is expected, or (more commonly) the function would create the Future itself. Unless I'm totally missing the programming model here. PS. I'd like to learn more about ExceptionStackContext() -- I've struggled somewhat with getting decent tracebacks in NDB.

...

...

In Tornado the Future is created by a decorator and hidden from the asynchronous function (it just sees the callback),

Hm, interesting. NDB goes the other way, the callbacks are mostly used to make Futures work, and most code (including large swaths of internal code) uses Futures. I think NDB is similar to monocle here. In NDB, you can do

f = <some function returning a Future> r = yield f

where "yield f" is mostly equivalent to f.result(), except it gives better opportunity for concurrency.

Yes, tornado's gen.engine does the same thing here. However, the stakes are higher than "better opportunity for concurrency" - in an event loop if you call future.result() without yielding, you'll deadlock if that Future's task needs to run on the same event loop.

That would depend on the semantics of the event loop implementation. In NDB's event loop, such a .result() call would just recursively enter the event loop, and you'd only deadlock if you actually have two pieces of code waiting for each other's completion. [...]

...

I am currently trying to understand if using "yield from" (and returning a value from a generator) will simplify things. For example maybe the need for a special decorator might go away. But I keep getting headaches -- perhaps there's a Monad involved. :-)

I think if you build generator handling directly into the event loop and use "yield from" for calls from one async function to another then you can get by without any decorators. But I'm not sure if you can do that and maintain any compatibility with existing non-generator async code.

I think the ability to return from a generator is actually a bigger deal than "yield from" (and I only learned about it from another python-ideas thread today). The only reason a generator decorated with @tornado.gen.engine needs a callback passed in to it is to act as a psuedo-return, and a real return would prevent the common mistake of running the callback then falling through to the rest of the function.

Ah, so you didn't come up with the clever hack of raising an exception to signify the return value. In NDB, you raise StopIteration (though it is given the alias 'Return' for clarity) with an argument, and the wrapper code that is responsible for the Future takes the value from the StopIteration exception and passes it to the Future's set_result().

For concreteness, here's a crude sketch of what the APIs I'm talking about would look like in use (in a hypothetical future version of tornado).

@future_wrap @gen.engine def async_http_client(url, callback): parsed_url = urlparse.urlsplit(url) # works the same whether the future comes from a thread pool or @future_wrap

And you need the thread pool because there's no async version of getaddrinfo(), right?

addrinfo = yield g_thread_pool.submit(socket.getaddrinfo, parsed_url.hostname, parsed_url.port) stream = IOStream(socket.socket()) yield stream.connect((addrinfo[0][-1])) stream.write('GET %s HTTP/1.0' % parsed_url.path)

Why no yield in front of the write() call?

header_data = yield stream.read_until('\r\n\r\n') headers = parse_headers(header_data) body_data = yield stream.read_bytes(int(headers['Content-Length'])) stream.close() callback(body_data)

# another function to demonstrate composability @future_wrap @gen.engine def fetch_some_urls(url1, url2, url3, callback): body1 = yield async_http_client(url1) # yield a list of futures for concurrency future2 = yield async_http_client(url2) future3 = yield async_http_client(url3) body2, body3 = yield [future2, future3] callback((body1, body2, body3))

This second one is nearly identical to the way we it's done in NDB. However I think you have a typo -- I doubt that there should be yields on the lines creating future2 and future3.

One hole in this design is how to deal with callbacks that are run multiple times. For example, the IOStream read methods take both a regular callback and an optional streaming_callback (which is called with each chunk of data as it arrives). I think this needs to be modeled as something like an iterator of Futures, but I haven't worked out the details yet.

Ah. Yes, that's a completely different kind of thing, and probably needs to be handled in a totally different way. I think it probably needs to be modeled more like an infinite loop where at the blocking point (e.g. a low-level read() or accept() call) you yield a Future. Although I can see that this doesn't work well with the IOLoop's concept of file descriptor (or other event source) registration. -- --Guido van Rossum (python.org/~guido)

Ben Darnell wrote:

StackContext doesn't quite give you better tracebacks, although I think it could be adapted to do that. ExceptionStackContext is essentially a try/except block that follows you around across asynchronous operations - on entry it sets a thread-local state, and all the tornado asynchronous functions know to save this state when they are passed a callback, and restore it when they execute it.

This is something that generator-based coroutines using yield-from ought to handle a lot more cleanly. You should be able to just use an ordinary try-except block in your generator code and have it do the right thing. I hope that the new async core will be designed so that generator-based coroutines can be plugged into it directly and efficiently, without the need for a lot of decorators, callbacks, Futures, etc. in between. -- Greg

Guido van Rossum wrote:

Indeed, in NDB this works great. However tracebacks don't work so great: If you don't catch the exception right away, it takes work to make the tracebacks look right when you catch it a few generator calls down on the (conceptual) stack. I fixed this to some extent in NDB, by passing the traceback explicitly along when setting an exception on a Future;

Was this before or after the recent change that was supposed to improve tracebacks from yield-fram chains? If there's still a problem after that, maybe exception handling in yield-from requires some more work.

But so far when thinking about this recently I have found the goal elusive --

Perhaps you can clear things up by showing some detailed (but still simple enough) example code to handle e.g. a simple web client?

You might like to take a look at this, where I develop a series of examples culminating in a simple multi-threaded server: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa... Code here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa...

somehow it seems there *has* to be a distinction between an operation you just *yield* (this would be waiting for a specific low-level I/O operation) and something you use with yield-from, which returns a value through StopIteration.

It may be worth noting that nothing in my server example uses 'yield' to send or receive values -- yield is only used without argument as a suspension point. But the functions containing the yields *are* called with yield-from and may return values via StopIteration. So I think there are (at least) two distinct ways of using generators, but the distinction isn't quite the one you're making. Rather, we have "coroutines" (don't yield values, do return values) and "iterators" (do yield values, don't return values). Moreover, it's *only* the "coroutine" variety that we need to cater for when designing an async event system. Does that help to alleviate any of your monad-induced headaches? -- Greg

Tue, Oct 9, 2012 at 5:44 PM, Greg Ewing wrote:

Guido van Rossum wrote:

...

Indeed, in NDB this works great. However tracebacks don't work so great: If you don't catch the exception right away, it takes work to make the tracebacks look right when you catch it a few generator calls down on the (conceptual) stack. I fixed this to some extent in NDB, by passing the traceback explicitly along when setting an exception on a Future;

Was this before or after the recent change that was supposed to improve tracebacks from yield-fram chains? If there's still a problem after that, maybe exception handling in yield-from requires some more work.

Sadly it was with Python 2.5/2.7...

...

But so far when thinking about this recently I have found the goal elusive --

...

Perhaps you can clear things up by

showing some detailed (but still simple enough) example code to handle e.g. a simple web client?

You might like to take a look at this, where I develop a series of examples culminating in a simple multi-threaded server:

http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa...

Definitely very enlightening. Though I think you should not use 'thread' since that term is already reserved for OS threads as supported by the threading module. In NDB I chose to use 'tasklet' -- while that also has other meanings, its meaning isn't fixed in core Python. You could also use task, which also doesn't have a core Python meaning. Just don't call it "process", never mind that Erlang uses this (a number of other languages rooted in old traditions do too, I believe). Also I think you can now revisit it and rewrite the code to use Python 3.3.

Code here:

http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/yf_current/Exa...

It does bother me somehow that you're not using .send() and yield arguments at all. I notice that you have a lot ofthree-line code blocks like this: block_for_reading(sock) yield data = sock.recv(1024) The general form seems to be: arrange for a callback when some operation can be done without blocking yield do the operation This seems to be begging to be collapsed into a single line, e.g. data = yield sock.recv_async(1024) (I would also prefer to see the socket wrapped in an object that makes it hard to accidentally block.)

...

somehow it seems there *has* to be a distinction between an operation you just *yield* (this would be waiting for a specific low-level I/O operation) and something you use with yield-from, which returns a value through StopIteration.

It may be worth noting that nothing in my server example uses 'yield' to send or receive values -- yield is only used without argument as a suspension point. But the functions containing the yields *are* called with yield-from and may return values via StopIteration.

Yeah, but see my remark above...

So I think there are (at least) two distinct ways of using generators, but the distinction isn't quite the one you're making. Rather, we have "coroutines" (don't yield values, do return values) and "iterators" (do yield values, don't return values).

But surely there's still a place for send() and other PEP 342 features?

Moreover, it's *only* the "coroutine" variety that we need to cater for when designing an async event system. Does that help to alleviate any of your monad-induced headaches?

Not entirely, no. I now have a fair amount experience writing an async system and helping users make sense of its error messages, and there are some practical considerations. E.g. my users sometimes want to treat something as a coroutine but they don't have any yields in it (perhaps they are writing skeleton code and plan to fill in the I/O later). Example: def caller(): data = yield from reader() def reader(): return 'dummy' yield works, but if you drop the yield it doesn't work. With a decorator I know how to make it work either way. -- --Guido van Rossum (python.org/~guido)

Guido van Rossum wrote:

Though I think you should not use 'thread' since that term is already reserved for OS threads as supported by the threading module. ... You could also use task, which also doesn't have a core Python meaning.

Also I think you can now revisit it and rewrite the code to use Python 3.3.

Both good ideas. I'll see about publishing an updated version.

It does bother me somehow that you're not using .send() and yield arguments at all. I notice that you have a lot ofthree-line code blocks like this:

block_for_reading(sock) yield data = sock.recv(1024)

I wouldn't say I have a "lot". In the spamserver, there are really only three -- one for accepting a connection, one for reading from a socket, and one for writing to a socket. These are primitive operations that would be provided by an async socket library. Generally, all the yields would be hidden inside primitives like this. Normally, user code would never need to use 'yield', only 'yield from'. This probably didn't come through as clearly as it might have in my tutorial. Part of the reason is that at the time I wrote it, I was having to manually expand yield-froms into for-loops, so I was reluctant to use any more of them than I needed to. Also, yield-from was a new and unfamiliar concept, and I didn't want to scare people by overusing it. These considerations led me to push some of the yields slightly further up the layer stack than they could be.

The general form seems to be:

arrange for a callback when some operation can be done without blocking yield do the operation

This seems to be begging to be collapsed into a single line, e.g.

data = yield sock.recv_async(1024)

I'm not sure how you're imagining that would work, but whatever it is, it's wrong -- that just doesn't make sense. What *would* make sense is data = yield from sock.recv_async(1024) with sock.recv_async() being a primitive that encapsulates the block/yield/process triplet.

(I would also prefer to see the socket wrapped in an object that makes it hard to accidentally block.)

It would be straightforward to make the primitives be methods of a socket wrapper object. I only used functions in the tutorial in the interests of keeping the amount of machinery to a bare minimum.

But surely there's still a place for send() and other PEP 342 features?

In the wider world of generator usage, yes. If you have a generator that it makes sense to send() things into, for example, and you want to factor part of it out into another function, the fact that yield-from passes through sent values is useful. But we're talking about a very specialised use of generators here, and so far I haven't thought of a use for sent or yielded values in this context that can't be done in a more straightforward way by other means. Keep in mind that a value yielded by a generator being used as part of a coroutine is *not* seen by code calling it with yield-from. Rather, it comes out in the inner loop of the scheduler, from the next() call being used to resume the coroutine. Likewise, any send() call would have to be made by the scheduler, not the yield-from caller. So, the send/yield channel is exclusively for communication with the *scheduler* and nothing else. Under the old way of doing generator-based coroutines, this channel was used to simulate a call stack by yielding 'call' and 'return' instructions that the scheduler interpreted. But all that is now taken care of by the yield-from mechanism, and there is nothing left for the send/yield channel to do.

my users sometimes want to treat something as a coroutine but they don't have any yields in it

def caller(): data = yield from reader()

def reader(): return 'dummy' yield

works, but if you drop the yield it doesn't work. With a decorator I know how to make it work either way.

If you're talking about a decorator that turns a function into a generator, I can't see anything particularly headachish about that. If you mean something else, you'll have to elaborate. -- Greg

On Mon, Oct 8, 2012 at 10:12 PM, Ben Darnell wrote:

On Mon, Oct 8, 2012 at 8:30 AM, Guido van Rossum wrote:

...

...

It's a Future constructor, a (conditional) add_done_callback, plus the calls to set_result or set_exception and the with statement for error handling. In full:

def future_wrap(f): @functools.wraps(f) def wrapper(*args, **kwargs): future = Future() if kwargs.get('callback') is not None: future.add_done_callback(kwargs.pop('callback')) kwargs['callback'] = future.set_result def handle_error(typ, value, tb): future.set_exception(value) return True with ExceptionStackContext(handle_error): f(*args, **kwargs) return future return wrapper

Hmm... I *think* it automatically adds a special keyword 'callback' to the *call* site so that you can do things like

fut = some_wrapped_func(blah, callback=my_callback)

and then instead of using yield to wait for the callback, put the continuation of your code in the my_callback() function.

Yes. Note that if you're passing in a callback you're probably going to just ignore the return value. The callback argument and the future return value are essentially two alternative interfaces; it probably doesn't make sense to use both at once (but as a library author it's useful to provide both).

Definitely sounds like something that could be simplified if you didn't have backward compatibility baggage...

...

But it also seems like it passes callback=future.set_result as the callback to the wrapped function, which looks to me like that function was apparently written before Futures were widely used. This seems pretty impure to me and I'd like to propose a "future" where such functions either be given the Future where the result is expected, or (more commonly) the function would create the Future itself.

Yes, it's impure and based on pre-Future patterns. The caller's callback argument and the inner function's callback not really related any more (they were the same in pre-Future async code of course). They should probably have different names, although if the inner function's return value were passed via exception (StopIteration or return) the inner callback argument can just go away.

...

Unless I'm totally missing the programming model here.

PS. I'd like to learn more about ExceptionStackContext() -- I've struggled somewhat with getting decent tracebacks in NDB.

StackContext doesn't quite give you better tracebacks, although I think it could be adapted to do that. ExceptionStackContext is essentially a try/except block that follows you around across asynchronous operations - on entry it sets a thread-local state, and all the tornado asynchronous functions know to save this state when they are passed a callback, and restore it when they execute it. This has proven to be extremely helpful in ensuring that all exceptions get caught by something that knows how to do the appropriate cleanup (i.e. an asynchronous web page serves an error instead of just spinning forever), although it has turned out to be a little more intrusive and magical than I had originally anticipated.

https://github.com/facebook/tornado/blob/master/tornado/stack_context.py

Heh. I'll try to mine it for gems.

...

...

...

...

In Tornado the Future is created by a decorator and hidden from the asynchronous function (it just sees the callback),

Hm, interesting. NDB goes the other way, the callbacks are mostly used to make Futures work, and most code (including large swaths of internal code) uses Futures. I think NDB is similar to monocle here. In NDB, you can do

f = <some function returning a Future> r = yield f

where "yield f" is mostly equivalent to f.result(), except it gives better opportunity for concurrency.

Yes, tornado's gen.engine does the same thing here. However, the stakes are higher than "better opportunity for concurrency" - in an event loop if you call future.result() without yielding, you'll deadlock if that Future's task needs to run on the same event loop.

That would depend on the semantics of the event loop implementation. In NDB's event loop, such a .result() call would just recursively enter the event loop, and you'd only deadlock if you actually have two pieces of code waiting for each other's completion.

Hmm, I think I'd rather deadlock. :) If the event loop is reentrant then the application code has be coded defensively as if it were preemptively multithreaded, which introduces the possibility of deadlock or (probably) more subtle/less frequent errors. Reentrancy has been a significant problem in my experience, so I've been moving towards a policy where methods in Tornado that take a callback never run it immediately; callbacks are always scheduled on the next iteration of the IOLoop with IOLoop.add_callback.

The latter is a good tactic and I'm also using it. (Except for some reason we had to add the concept of "immediate callbacks" to our Future class, and those are run inside the set_result() call. But most callbacks don't use that feature.) I don't have a choice about making the event loop reentrant -- App Engine's underlying RPC multiplexing implementation *is* reentrant, and there is a large set of "classic" APIs that I cannot stop the user from calling that reenter it. But even if my hand wasn't forced, I'm not sure if I would make your choice. In NDB, there is a full complement of synchronous APIs that exactly matches the async APIs, and users are free to use the synchronous APIs in parts of their code where they don't need concurrency. Hence, every sychronous API just calls its async sibling and immediately waits for its result, which implicitly invokes the event loop. Of course, I have it easy -- multiple incoming requests are dispatched to separate threads by the App Engine runtime, so I don't have to worry about multiplexing at that level at all -- just end user code that is essentially single-threaded unless they go out of their way. I did end up debugging one user's problem where they were making a synchronous call inside an async handler, and -- very rarely! -- the recursive event loop calls kept stacking up until they hit a StackOverflowError. So I would agree that async code shouldn't make synchronous API calls; but I haven't heard yet from anyone who was otherwise hurt by the recursive event loop invocations -- in particular, nobody has requested locks. Still, this sounds like an important issue to revisit when discussing a standard reactor API as part of Lourens's PEP offensive.

...

[...]

...

...

I am currently trying to understand if using "yield from" (and returning a value from a generator) will simplify things. For example maybe the need for a special decorator might go away. But I keep getting headaches -- perhaps there's a Monad involved. :-)

I think if you build generator handling directly into the event loop and use "yield from" for calls from one async function to another then you can get by without any decorators. But I'm not sure if you can do that and maintain any compatibility with existing non-generator async code.

I think the ability to return from a generator is actually a bigger deal than "yield from" (and I only learned about it from another python-ideas thread today). The only reason a generator decorated with @tornado.gen.engine needs a callback passed in to it is to act as a psuedo-return, and a real return would prevent the common mistake of running the callback then falling through to the rest of the function.

Ah, so you didn't come up with the clever hack of raising an exception to signify the return value. In NDB, you raise StopIteration (though it is given the alias 'Return' for clarity) with an argument, and the wrapper code that is responsible for the Future takes the value from the StopIteration exception and passes it to the Future's set_result().

I think I may have thought about "raise Return(x)" and dismissed it as too weird. But then, I'm abnormally comfortable with asynchronous code that passes callbacks around.

As I thought about the issue of how to spell "return a value" and looked at various approaches, I decided I definitely didn't like what monocle does: they let you say "yield X" where X is a non-Future value; and I saw some other solution (Twisted? Phillip Eby?) that simply called a function named something like returnValue(X). But I also wanted it to look like a control statement that ends a block (so auto-indenting editors would auto-dedent the next line), and that means there are only four choices: continue, break, raise or return. Three of those are useless... So the only choice really was which exception to raise. FOrtunately I had the advantage of knowing that PEP 380 was going to implement "return X" from a generator as "raise StopIteration(X)" so I decided to be compatible with that.

...

...

For concreteness, here's a crude sketch of what the APIs I'm talking about would look like in use (in a hypothetical future version of tornado).

@future_wrap @gen.engine def async_http_client(url, callback): parsed_url = urlparse.urlsplit(url) # works the same whether the future comes from a thread pool or @future_wrap

And you need the thread pool because there's no async version of getaddrinfo(), right?

Right.

...

...

addrinfo = yield g_thread_pool.submit(socket.getaddrinfo, parsed_url.hostname, parsed_url.port) stream = IOStream(socket.socket()) yield stream.connect((addrinfo[0][-1])) stream.write('GET %s HTTP/1.0' % parsed_url.path)

Why no yield in front of the write() call?

Because we don't need to wait for the write to complete before we continue to the next statement. write() doesn't return anything; it just succeeds or fails, and if it fails the next read_until will fail too. (although in this case it wouldn't hurt to have the yield either)

I guess you have a certain kind of buffering built in to your stream? So if you make two write() calls without waiting in quick succession, does the system collapse these into one, or does it end up making two system calls, or what? In NDB, there's a similar issue with multiple RPCs that can be batched. I ended up writing an abstraction that automatically combines these; the call isn't actually made until there are no other runnable tasks. I've had to explain this a few times to users who try to get away with overlapping CPU work and I/O, but otherwise it's worked quite well.

...

...

header_data = yield stream.read_until('\r\n\r\n') headers = parse_headers(header_data) body_data = yield stream.read_bytes(int(headers['Content-Length'])) stream.close() callback(body_data)

# another function to demonstrate composability @future_wrap @gen.engine def fetch_some_urls(url1, url2, url3, callback): body1 = yield async_http_client(url1) # yield a list of futures for concurrency future2 = yield async_http_client(url2) future3 = yield async_http_client(url3) body2, body3 = yield [future2, future3] callback((body1, body2, body3))

This second one is nearly identical to the way we it's done in NDB. However I think you have a typo -- I doubt that there should be yields on the lines creating future2 and future3.

Right.

...

...

One hole in this design is how to deal with callbacks that are run multiple times. For example, the IOStream read methods take both a regular callback and an optional streaming_callback (which is called with each chunk of data as it arrives). I think this needs to be modeled as something like an iterator of Futures, but I haven't worked out the details yet.

Ah. Yes, that's a completely different kind of thing, and probably needs to be handled in a totally different way. I think it probably needs to be modeled more like an infinite loop where at the blocking point (e.g. a low-level read() or accept() call) you yield a Future. Although I can see that this doesn't work well with the IOLoop's concept of file descriptor (or other event source) registration.

It works just fine at the IOLoop level: you call IOLoop.add_handler(fd, func, READ), and you'll get read events whenever there's new data until you call remove_handler(fd) (or update_handler). If you're passing callbacks around explicitly it's pretty straightforward (as much as anything ever is in that style) to allow for those callbacks to be run more than once. The problem is that generators more or less require that each callback be run exactly once. That's a generally desirable property, but the mismatch between the two layers can be difficult to deal with.

Okay, I see that these are useful. However they feel as two very different classes of callbacks -- one that is called when a *specific* piece of I/O that was previously requested is done; another that will be called *whenever* a certain condition becomes true on a certain channel. The former would correspond to e.g. completion of the headers of an incoming HTTP request); the latter might correspond to a "listening" socket receiving another connection. -- --Guido van Rossum (python.org/~guido)

On Thu, Oct 11, 2012 at 5:41 PM, Ben Darnell wrote:

On Thu, Oct 11, 2012 at 3:28 PM, Guido van Rossum wrote:

...

On Mon, Oct 8, 2012 at 10:12 PM, Ben Darnell wrote:

...

On Mon, Oct 8, 2012 at 8:30 AM, Guido van Rossum wrote:

...

...

It's a Future constructor, a (conditional) add_done_callback, plus the calls to set_result or set_exception and the with statement for error handling. In full:

def future_wrap(f): @functools.wraps(f) def wrapper(*args, **kwargs): future = Future() if kwargs.get('callback') is not None: future.add_done_callback(kwargs.pop('callback')) kwargs['callback'] = future.set_result def handle_error(typ, value, tb): future.set_exception(value) return True with ExceptionStackContext(handle_error): f(*args, **kwargs) return future return wrapper

Hmm... I *think* it automatically adds a special keyword 'callback' to the *call* site so that you can do things like

fut = some_wrapped_func(blah, callback=my_callback)

and then instead of using yield to wait for the callback, put the continuation of your code in the my_callback() function.

Yes. Note that if you're passing in a callback you're probably going to just ignore the return value. The callback argument and the future return value are essentially two alternative interfaces; it probably doesn't make sense to use both at once (but as a library author it's useful to provide both).

Definitely sounds like something that could be simplified if you didn't have backward compatibility baggage...

Probably, although I still feel like callback-passing has its place. For example, I think the Tornado chat demo (https://github.com/facebook/tornado/blob/master/demos/chat/chatdemo.py) would be less clear with coroutines and Futures than it is now (although it would fit better into Greg's schedule/unschedule style).

Hmm... That's an interesting challenge. I can't quite say I understand that whole program yet, but I'd like to give it a try. I think it can be made clearer than Tornado with Futures and coroutines -- it all depends on how you define your primitives.

That doesn't mean that every method has to take a callback, but I'd be reluctant to get rid of them until we have more experience with the generator/future-focused style.

Totally understood. Though the nice thing of Futures is that you can tie callbacks to them *or* use them in coroutines.

...

...

...

...

...

> In Tornado the Future is created by a decorator > and hidden from the asynchronous function (it just sees the callback),

Hm, interesting. NDB goes the other way, the callbacks are mostly used to make Futures work, and most code (including large swaths of internal code) uses Futures. I think NDB is similar to monocle here. In NDB, you can do

f = <some function returning a Future> r = yield f

where "yield f" is mostly equivalent to f.result(), except it gives better opportunity for concurrency.

Yes, tornado's gen.engine does the same thing here. However, the stakes are higher than "better opportunity for concurrency" - in an event loop if you call future.result() without yielding, you'll deadlock if that Future's task needs to run on the same event loop.

That would depend on the semantics of the event loop implementation. In NDB's event loop, such a .result() call would just recursively enter the event loop, and you'd only deadlock if you actually have two pieces of code waiting for each other's completion.

Hmm, I think I'd rather deadlock. :) If the event loop is reentrant then the application code has be coded defensively as if it were preemptively multithreaded, which introduces the possibility of deadlock or (probably) more subtle/less frequent errors. Reentrancy has been a significant problem in my experience, so I've been moving towards a policy where methods in Tornado that take a callback never run it immediately; callbacks are always scheduled on the next iteration of the IOLoop with IOLoop.add_callback.

The latter is a good tactic and I'm also using it. (Except for some reason we had to add the concept of "immediate callbacks" to our Future class, and those are run inside the set_result() call. But most callbacks don't use that feature.)

I don't have a choice about making the event loop reentrant -- App Engine's underlying RPC multiplexing implementation *is* reentrant, and there is a large set of "classic" APIs that I cannot stop the user from calling that reenter it. But even if my hand wasn't forced, I'm not sure if I would make your choice. In NDB, there is a full complement of synchronous APIs that exactly matches the async APIs, and users are free to use the synchronous APIs in parts of their code where they don't need concurrency. Hence, every sychronous API just calls its async sibling and immediately waits for its result, which implicitly invokes the event loop.

Tornado has a synchronous HTTPClient that does the same thing, although each fetch creates and runs its own IOLoop rather than spinning the top-level IOLoop. (This means it doesn't really make sense to run it when there is a top-level IOLoop; it's provided as a convenience for scripts and multi-threaded apps who want an HTTPRequest interface consistent with the async version).

I see. Yet another possible design choice.

...

Of course, I have it easy -- multiple incoming requests are dispatched to separate threads by the App Engine runtime, so I don't have to worry about multiplexing at that level at all -- just end user code that is essentially single-threaded unless they go out of their way.

I did end up debugging one user's problem where they were making a synchronous call inside an async handler, and -- very rarely! -- the recursive event loop calls kept stacking up until they hit a StackOverflowError. So I would agree that async code shouldn't make synchronous API calls; but I haven't heard yet from anyone who was otherwise hurt by the recursive event loop invocations -- in particular, nobody has requested locks.

I think that's because you don't have file descriptor support. In a (level-triggered) event loop if you don't drain the socket before reentering the loop then your read handler will be called again, which generally makes a mess. I suppose with coroutines you'd want edge-triggered instead of level-triggered though, which might make this problem go away.

Ah, good terminology. Coroutines definitely like being edge-triggered.

...

...

...

...

For concreteness, here's a crude sketch of what the APIs I'm talking about would look like in use (in a hypothetical future version of tornado).

@future_wrap @gen.engine def async_http_client(url, callback): parsed_url = urlparse.urlsplit(url) # works the same whether the future comes from a thread pool or @future_wrap

And you need the thread pool because there's no async version of getaddrinfo(), right?

Right.

...

...

addrinfo = yield g_thread_pool.submit(socket.getaddrinfo, parsed_url.hostname, parsed_url.port) stream = IOStream(socket.socket()) yield stream.connect((addrinfo[0][-1])) stream.write('GET %s HTTP/1.0' % parsed_url.path)

Why no yield in front of the write() call?

Because we don't need to wait for the write to complete before we continue to the next statement. write() doesn't return anything; it just succeeds or fails, and if it fails the next read_until will fail too. (although in this case it wouldn't hurt to have the yield either)

I guess you have a certain kind of buffering built in to your stream? So if you make two write() calls without waiting in quick succession, does the system collapse these into one, or does it end up making two system calls, or what? In NDB, there's a similar issue with multiple RPCs that can be batched. I ended up writing an abstraction that automatically combines these; the call isn't actually made until there are no other runnable tasks. I've had to explain this a few times to users who try to get away with overlapping CPU work and I/O, but otherwise it's worked quite well.

Yes, IOStream does buffering for you. Each IOStream.write() call will generally result in a syscall, but once the outgoing socket buffer is full subsequent writes will be buffered in the IOStream and written when the IOLoop says the socket is writable. (the callback argument to write() can be used for flow control in this case) I used to defer the syscall until the IOLoop was idle to batch things up, but it turns out to be more efficient in practice to just write things out each time and let the higher level do its own buffering when appropriate.

Makes sense. I think different people might want to implement slightly different IOStream-like abstractions; this would be a good test of the infrastructure. You should be able to craft one from scratch out of sockets and Futures, but there should be one or two standard ones as well, and they should all happily mix and match using the same reactor. -- --Guido van Rossum (python.org/~guido)

On Sun, Oct 7, 2012 at 9:01 PM, Guido van Rossum wrote:

On Sun, Oct 7, 2012 at 6:41 PM, Ben Darnell wrote:

...

I think there are actually two separate issues here and it's important to keep them distinct: at a low level, there is a need for a standardized event loop, while at a higher level there is a question of what asynchronous code should look like.

Yes, yes. I tried to bring up thing distinction. I'm glad I didn't completely fail.

Perhaps this is obvious to others, but (like hinted at above) there seem to be two primary issues with event handlers: 1) event handlers for the machine-program interface (ex. network I/O) 2) event handlers for the program-user interface (ex. mouse I/O) While similar, my gut tell me they have to be handled in completely different way in order to preserve order (i.e. sanity). This issue, for me, has come up with wanting to make a p2p network application with VPython. MarkJ

Mark Adam wrote:

1) event handlers for the machine-program interface (ex. network I/O) 2) event handlers for the program-user interface (ex. mouse I/O)

While similar, my gut tell me they have to be handled in completely different way in order to preserve order (i.e. sanity).

They can't be *completely* different, because deep down there has to be a single event loop that can handle all kinds of asynchronous events. Upper layers can provide different APIs for them, but there has to be some commonality in the lowest layers. -- Greg

On Tue, Oct 9, 2012 at 1:53 AM, Ben Darnell wrote:

On Mon, Oct 8, 2012 at 10:56 PM, Greg Ewing wrote:

...

Mark Adam wrote:

...

1) event handlers for the machine-program interface (ex. network I/O) 2) event handlers for the program-user interface (ex. mouse I/O)

While similar, my gut tell me they have to be handled in completely different way in order to preserve order (i.e. sanity).

They can't be *completely* different, because deep down there has to be a single event loop that can handle all kinds of asynchronous events.

There doesn't *have* to be - you could run a network event loop in one thread and a GUI event loop in another and pass control back and forth via methods like IOLoop.add_callback or Reactor.callFromThread.

No, this won't work. The key FAIL in that sentence is "...and pass control", because the O.S. has to be in charge of things that happen in user space. And everything in Python happens in user space. (hence my suggestion of creating a Python O.S.). MarkJ

...

...

...

Mark Adam wrote:

...

...

There doesn't *have* to be - you could run a network event loop in one thread and a GUI event loop in another and pass control back and forth via methods like IOLoop.add_callback or Reactor.callFromThread.

Well, that could be done, but one of the reasons for using an event loop approach in the first place is to avoid having to deal with threads and all their attendant concurrency problems. -- Greg

On Mon, Oct 8, 2012 at 5:39 AM, Christian Heimes wrote:

Python's standard library doesn't contain in interface to I/O Completion Ports. I think a common event loop system is a good reason to add IOCP if somebody is up for the challenge.

Would you prefer an IOCP wrapper in the stdlib or your own version? Twisted has its own Cython based wrapper, some other libraries use a libevent-based solution.

What's an IOCP? -- --Guido van Rossum (python.org/~guido)

On 8 October 2012 18:04, Mike Graham wrote:

...

What's an IOCP?

It's the non-crappy select equivalent on Windows.

I/O Completion port, just for clarity :-) Paul.

On Mon, Oct 8, 2012 at 11:36 AM, Antoine Pitrou wrote:

On Mon, 8 Oct 2012 13:04:00 -0400 Mike Graham wrote:

...

On Mon, Oct 8, 2012 at 11:35 AM, Guido van Rossum wrote:

...

On Mon, Oct 8, 2012 at 5:39 AM, Christian Heimes wrote:

...

Python's standard library doesn't contain in interface to I/O Completion Ports. I think a common event loop system is a good reason to add IOCP if somebody is up for the challenge.

Would you prefer an IOCP wrapper in the stdlib or your own version? Twisted has its own Cython based wrapper, some other libraries use a libevent-based solution.

What's an IOCP?

It's the non-crappy select equivalent on Windows.

Except that it's not exactly an equivalent, it's a whole different programming model ;)

(but I understand what you mean: it allows to do non-blocking I/O on an arbitrary number of objects in parallel)

Now I know what it is I think that (a) the abstract reactor design should support IOCP, and (b) the stdlib should have enabled by default IOCP when on Windows. -- --Guido van Rossum (python.org/~guido)

Am 08.10.2012 17:35, schrieb Guido van Rossum:

On Mon, Oct 8, 2012 at 5:39 AM, Christian Heimes wrote:

...

Python's standard library doesn't contain in interface to I/O Completion Ports. I think a common event loop system is a good reason to add IOCP if somebody is up for the challenge.

Would you prefer an IOCP wrapper in the stdlib or your own version? Twisted has its own Cython based wrapper, some other libraries use a libevent-based solution.

What's an IOCP?

I/O Completion Ports, http://en.wikipedia.org/wiki/IOCP It's a Windows (and apparently also Solaris) API for async IO that can handle multiple threads. Christian

On Wed, 10 Oct 2012 20:55:23 -0400 Trent Nelson wrote:

You could leverage this with kqueue and epoll; have similar threads set up to simply process I/O independent of the GIL, using the same facilities that would be used by IOCP-processing threads.

Would you really win anything by doing I/O in separate threads, while doing normal request processing in the main thread? That said, the idea of a common API architected around async I/O, rather than non-blocking I/O, sounds interesting at least theoretically. Maybe all those outdated Snakebite Operating Systems are useful for something after all. ;-P cheers Antoine. -- Software development and contracting: http://pro.pitrou.net