On Feb 10, 2020, at 02:34, Bar Harel <bharel@barharel.com> wrote:
While I'm not this idea can totally work, I do think we need a method of closing the gap between asyc coroutines and normal functions.
As of today, async applications need to create an entirely different codebase, sometimes just by copy and paste, while changing function definitions to async and function calls to await.
This creates a lot of work, from porting code to maintaining both variants, to just having code duplication all around.
It’s a well-known problem that async is “contagious”: once you turn one function into a coroutine, you have to turn the whole call tree async up to some point where you can either block or daemonize a coroutine. Together with the fact that async operations have not just different syntax but different names and APIs than their sync equivalents, this makes it hard to port code between the two paradigms. But C# and every other language that’s borrowed the idea has the same problem, and as far as I know, nobody’s thought of a good answer yet. Just saying “this is bad, so we should do something” doesn’t get us anywhere unless that something is doable and solves the problem and we know what it is. And I don’t know if that’s the case here. The OP proposed something that can only work by impossible magic, and you suggested it would be nice if we had something actually implementable that was just as easy to use, and it definitely would be nice, but unless someone has an idea for that something… (That definitely doesn’t mean it isn’t worth pursuing. Just because nobody else has thought of it yet doesn’t mean it’s impossible, after all.) I think breaking down decorators into smaller building blocks that can be composed might be promising. If you have a 30 different decorators that all want to do something (that doesn’t block) before calling the wrapper function or coro, you can write a single `@before` that does have to duplicate its code and then 30 decorators that use `before` and don’t need any duplication. I’m not sure how far you can get with that. Certainly not all the way to the full generality of arbitrary-code decorators (although I don’t think even the OP’s magic would get use there—you still can’t put any code into the decorator that might be blocking except for the wrapped function, especially since usually the difference isn’t just a matter of await or not, but of calling a different method on a different object…). But maybe far enough to be useful? I don’t know. Another possible avenue is to drop back a level and write wrappers that work on futures. Async futures and concurrent futures duck type as lowest-common-denominator futures, and maybe some of the helpers you want can be built that way without duplicating their code? Meanwhile, for porting large existing threaded or synchronous codebases, you may be better off using the gevent model, where awaiting is internal and invisible. You can find out pretty quickly whether all of your libraries are compatible or not. If they are, you won’t get all of the advantages (e.g., with await it’s often obvious when you can remove a mutex or other sync object; with gevent, because it looks nearly identical to preemptive threading, if that mutex was necessary in the preemptive version, it still looks necessary in the gevent version even though it isn’t, unless you think it through carefully), but only having to rewrite 2% of your code instead of 60% may be worth it anyway. Just because asyncio is usually the best model for new code doesn’t mean it’s the always best model for porting all code.