[Async-sig] Optional async method and best practices

[Laurent Mazuel]

Thanks Dima, Chris and Cory! This helps me a lot 😊

I like the "future" approach, I think this is exactly what I need, more than a coroutine method "result()/ready()" on my object. With a coroutine method, each time you call it you get a new coroutine, but it will poll the same set of values and it's pointless and waste of I/O+CPU. With a future, I control the coroutine to be sure there is only one (that I schedule with ensure_future).

I also agree that if the user doesn't want the poll result, save I/O and not poll at all. Using "resource(_nowait)", or a "nowait=True" keyword-only argument, I'm not sure yet. I was also thinking lazy initialization using a property, something like:

    def __init__(self):
        self._future = None

    async def _poll(self):
        self._future.set_result('Future is Done!')

    @property
    def future(self):
        if self._future is None:
            self._future = asyncio.Future()
            asyncio.ensure_future(self._poll())
        return self._future

result = await poller.future

If I don't call the "future" attribute, I don't poll at all. My initial "create" method returns the same object anytime, and I don't need a parameter or another "nowait" method. Do you have any caveat for issues I don’t see in this approach?

Thank you very much!!!

Laurent

PS: Cory, I just subscribed to the mailing list 😊

-----Original Message-----
From: Cory Benfield [mailto:cory at lukasa.co.uk] 
Sent: Wednesday, July 12, 2017 01:18
To: Laurent Mazuel <lmazuel at microsoft.com>
Cc: async-sig at python.org
Subject: Re: [Async-sig] Optional async method and best practices


> On 11 Jul 2017, at 23:26, Laurent Mazuel via Async-sig <async-sig at python.org> wrote:
> 
> Hello,

Hi Laurent! A future note: your message got stuck in moderation because you aren’t subscribed to the mailing list. You may find it helpful to subscribe, as your future messages will also get stuck unless you do!

> My first prototype was to split and return a tuple (resource, coroutine):
> 
>    obj, optional_poller = client.create(**parameters)
>    obj = await optional_poller # OPTIONAL
> 
> But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited 
> 
> I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.

The assumption in asyncio, generally speaking, is that you do not create coroutines you do not care about running. This is both for abstract theoretical reasons (if you don’t care if the coroutine is run or not, why not just optimise your code to never create the coroutine and save yourself the CPU cycles?) and for more concrete practical concerns (coroutines may own system resources and do cleanup in `finally` blocks, and if you don’t await a coroutine then you’ll never reach the `finally` block and so will leak system resources).

Given that there’s no computational way to be sure that *not* running a coroutine is safe (hello there halting problem), asyncio takes the pedantic model and says that not running a coroutine is a condition that justifies a warning. I think asyncio’s position here is correct, incidentally.

> 2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
> 
>    async_poller = client.create(**parameters)
>    obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. 
>    obj = await async_poller.result() # OPTIONAL
> 
> My async_poller object being something like:
> 
>    class PollerOperation:
>         async def result(self):
>              ...async version of previous sync result()...
> 
> So the questions are:
> - Does this seem a correct pattern?

Yes. This is the simple map to your old API, and is absolutely what I’d recommend doing in the first instance if you want to use coroutines.


> - Is there a simple way to achieve something like this:
> 
>    obj = await async_poller
> 
> meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊

There are a few other patterns you could use.

The first is to return a Future, and just always run the “polling” function in the background to resolve that future. If the caller doesn’t care about the result they can just ignore the Future, and if they do care they can await on it. This has the downside of always requiring the I/O to poll, but is otherwise pretty clean.

Another option is to offer two functions, for example `def resource_nowait()` and `async def resource`. The caller can decide which they want to call based on whether they care about finding out the result. This is the clearest approach that doesn’t trigger automatic extra work like the Future does, and it lacks magic: it’s very declarative. This is a nice approach for keeping things clean.

Finally, you can create a magic awaitable object. PEP 492 defines several ways to create an “awaitable”, but one approach is to use what it calls a “Future-like object”: that is, one with a __await__ method that returns an iterator. In this case, you’d do a very basic extension of the Future object by triggering the work upon the call of __await__ before delegating to the normal behaviour. This is an annoyingly precise thing to do, though technically do-able.

Cory