On Fri, Oct 12, 2012 at 4:39 PM, Richard Oudkerk <shibturn@gmail.com> wrote:
On 12/10/2012 11:11pm, Guido van Rossum wrote:
Using Futures and generator coroutines, I would do it as follows. I'm hypothesizing that for every blocking API foo() there is a corresponding non-blocking API foo_async() with the same call signature, and returning a Future whose result is what the synchronous API returns (and raises what the synchronous call would raise, if there's an error). These are the conventions I use in NDB. I'm also inventing a @task decorator.
@task def view_paste_async(request, filekey): # Create Futures -- no yields! f1 = Pastes.objects.get_async(key=filekey) # This won't raise f2 = loader.get_template_async('pastebin/error.html') f3 = loader.get_template_async('pastebin/paste.html')
try: fileinfo= yield f1 except DoesNotExist: t = yield f2 return HttpResponse(t.render(Context(dict(error='File does not exist'))))
f = yield open_async(fileinfo.filename) fcontents = yield f.read_async() t = yield f3 return HttpResponse(t.render(Context(dict(file=fcontents))))
So would the futures be registered with the reactor as soon as they are created, or only when they are yielded? I can't see how there can be any "concurrency" if they don't start till they are yielded. It would be like doing
t1 = Thread(target=f1) t2 = Thread(target=f2) t3 = Thread(target=f3) t1.start() t1.join() t2.start() t2.join() t3.start() t3.join()
But if the futures are registered immediately with the reactor then does that mean there is a singleton reactor? That seems rather inflexible.
I don't think it follows that there can only be one reactor if they are registered immediately. There could be a notion of "current reactor" maintained in thread-local context; moreover it could depend on the reactor that made the callback that caused the current task to run. The reactor could also be chosen by the code that made the Future. (Though I'm not immediately sure how that would work in the yield-from scenario -- but I'm sure there's a way.) FWIW, in NDB there is one event loop per thread; separate threads are handling separate requests and are completely independent. Also, in NDB there's some code that turns Futures into actual RPCs that runs only once there are no more immediately runnable tasks. I think that in general such behaviors are up to the reactor implementation for the platform though, and should not directly be reflected in the reactor API. -- --Guido van Rossum (python.org/~guido)