That's a great way of thinking about async structure - and perhaps surprisingly (since switching from asyncio or trying fledgling implementations as part of my day job is a no-go ;) immediately useful. A large part of what I do is wrap existing libraries so they can be used with asyncio- the idea being that once wrapped correctly it becomes easy to throw together applications quickly and correctly.  For example 'cassandra' or the python gRPC libraries. These both offer async-ish style APIs, smattered with some callback style stuff and a handful of functions that block but probably shouldn't. There are no standard examples of 'best practice' in how to do this - the asyncio docs focus on using existing asyncio components. As a result I end up with stuff that works until I need to worry about error handling, cancellation, restarting components and then I cry my heart out in the mailing lists and generally make a mess.

My key takeaway after skimming your blogs is that implementations should 'respect causality'. Aim for await my_implementation() to not spawn any anonymous tasks that can't be controlled, to only complete when it really has finished everything it started under the hood, and to correctly respect cancellation. Limit APIs to coroutines only (ie limit yourself to a 'curio' style) to make things simpler to reason about. If you must spawn tasks, keep them in logical groups - eg within a single function (or nursery if you have such an implementation) and make sure they are all finished before the function ends.

It seems to me like these are good guiding principles to knock together robust async/await APIs. At any rate, I'll keep them in mind and see if my next attempt end up with less subtle problems to worry about.

Thanks - and I look forward to really getting to grips with the detail of asynchronous design!

On Tue, 26 Feb 2019 at 12:14, Nathaniel Smith <njs@pobox.com> wrote:
On Mon, Feb 25, 2019 at 4:15 PM Josh Quigley <0zeroth@gmail.com> wrote:
>
> I've realised the error of my ways: because Task separates the scheduling from the response handling, you cannot know if an exception is unhandled until the task is deleted. So in my example the reference means the task is not deleted, so the exception is not yet unhandled.
>
> This is in contrast to APIs like call_soon(callable, success_callback, error_callback) where there the possibility of delayed error handling is not present. In that case the loop can reliably crash if either callback raises an exception.
>
> So, the 'solution' to this use-case is to always attach error handers to Tasks. A catch-all solution cannot catch every error case.

That's right. There are other ways to structure async code to avoid
running into these cases, that are implemented in Trio, and there are
discussions happening (slowly) about adding them into asyncio as well.
See:

https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Also, I could swear I saw some library that tried to implement
nurseries on asyncio, but I can't find it now... :-/ maybe someone
else here knows?

-n

--
Nathaniel J. Smith -- https://vorpus.org