Optional async method and best practices
Hello, I’m working currently with Brett Cannon to bring asyncio support to our SDK. We wanted to check with you one of the scenario, since we got a loooong discussion on it together 😊. And we want to do it using the best reasonable practice with your opinion. We have an api that is clearly async and will gain a lot to be converted to asyncio. However, it's a two-step operation. Operation 1 asks for the creation of a resource and is not async, operation 2 is *optional* and wait for completion of this creation (with nightmare threads currently and I removed a lot of code moving to asyncio - happiness). There is perfectly legit scenarios where operation 2 is not needed and avoid it is better, but it has to be prepared at the same time of operation 1. Current code looks like this: sync_poller = client.create(**parameters) obj = sync_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = sync_poller.result() # OPTIONAL. This is a blocking call with thread, if you want to wait for actual creation and get updated metadatas My first prototype was to split and return a tuple (resource, coroutine): obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly. I found 2 solutions to avoid the warning, and I currently prefer solution 2: 1- Return a function to call, and not a coroutine. The "await" statement becomes: obj = await optional_poller() 2- Return my initial object with an async method. This allows me to write (something finally close to the current code): async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL My async_poller object being something like: class PollerOperation: async def result(self): ...async version of previous sync result()... So the questions are: - Does this seem a correct pattern? - Is there a simple way to achieve something like this: obj = await async_poller meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊 If it helps, I'm willing to use 3.5 as minimal requirement to get async behavior. Thank you!! Laurent
On Tue, Jul 11, 2017 at 3:26 PM, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited ... I found 2 solutions to avoid the warning, and I currently prefer solution 2: 1- Return a function to call, and not a coroutine. The "await" statement becomes:
obj = await optional_poller()
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
Either of those options sounds fine to me. Instead of creating your coroutine object at the very beginning, create your coroutine *function*. Wait until you know you're going to do your second operation, and create your coroutine object then! --Chris
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern? - Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
If it helps, I'm willing to use 3.5 as minimal requirement to get async behavior.
Thank you!!
Laurent _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
On 11 Jul 2017, at 23:26, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
Hi Laurent! A future note: your message got stuck in moderation because you aren’t subscribed to the mailing list. You may find it helpful to subscribe, as your future messages will also get stuck unless you do!
My first prototype was to split and return a tuple (resource, coroutine):
obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited
I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.
The assumption in asyncio, generally speaking, is that you do not create coroutines you do not care about running. This is both for abstract theoretical reasons (if you don’t care if the coroutine is run or not, why not just optimise your code to never create the coroutine and save yourself the CPU cycles?) and for more concrete practical concerns (coroutines may own system resources and do cleanup in `finally` blocks, and if you don’t await a coroutine then you’ll never reach the `finally` block and so will leak system resources). Given that there’s no computational way to be sure that *not* running a coroutine is safe (hello there halting problem), asyncio takes the pedantic model and says that not running a coroutine is a condition that justifies a warning. I think asyncio’s position here is correct, incidentally.
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern?
Yes. This is the simple map to your old API, and is absolutely what I’d recommend doing in the first instance if you want to use coroutines.
- Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
There are a few other patterns you could use. The first is to return a Future, and just always run the “polling” function in the background to resolve that future. If the caller doesn’t care about the result they can just ignore the Future, and if they do care they can await on it. This has the downside of always requiring the I/O to poll, but is otherwise pretty clean. Another option is to offer two functions, for example `def resource_nowait()` and `async def resource`. The caller can decide which they want to call based on whether they care about finding out the result. This is the clearest approach that doesn’t trigger automatic extra work like the Future does, and it lacks magic: it’s very declarative. This is a nice approach for keeping things clean. Finally, you can create a magic awaitable object. PEP 492 defines several ways to create an “awaitable”, but one approach is to use what it calls a “Future-like object”: that is, one with a __await__ method that returns an iterator. In this case, you’d do a very basic extension of the Future object by triggering the work upon the call of __await__ before delegating to the normal behaviour. This is an annoyingly precise thing to do, though technically do-able. Cory
Thanks Dima, Chris and Cory! This helps me a lot 😊 I like the "future" approach, I think this is exactly what I need, more than a coroutine method "result()/ready()" on my object. With a coroutine method, each time you call it you get a new coroutine, but it will poll the same set of values and it's pointless and waste of I/O+CPU. With a future, I control the coroutine to be sure there is only one (that I schedule with ensure_future). I also agree that if the user doesn't want the poll result, save I/O and not poll at all. Using "resource(_nowait)", or a "nowait=True" keyword-only argument, I'm not sure yet. I was also thinking lazy initialization using a property, something like: def __init__(self): self._future = None async def _poll(self): self._future.set_result('Future is Done!') @property def future(self): if self._future is None: self._future = asyncio.Future() asyncio.ensure_future(self._poll()) return self._future result = await poller.future If I don't call the "future" attribute, I don't poll at all. My initial "create" method returns the same object anytime, and I don't need a parameter or another "nowait" method. Do you have any caveat for issues I don’t see in this approach? Thank you very much!!! Laurent PS: Cory, I just subscribed to the mailing list 😊 -----Original Message----- From: Cory Benfield [mailto:cory@lukasa.co.uk] Sent: Wednesday, July 12, 2017 01:18 To: Laurent Mazuel <lmazuel@microsoft.com> Cc: async-sig@python.org Subject: Re: [Async-sig] Optional async method and best practices
On 11 Jul 2017, at 23:26, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
Hi Laurent! A future note: your message got stuck in moderation because you aren’t subscribed to the mailing list. You may find it helpful to subscribe, as your future messages will also get stuck unless you do!
My first prototype was to split and return a tuple (resource, coroutine):
obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited
I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.
The assumption in asyncio, generally speaking, is that you do not create coroutines you do not care about running. This is both for abstract theoretical reasons (if you don’t care if the coroutine is run or not, why not just optimise your code to never create the coroutine and save yourself the CPU cycles?) and for more concrete practical concerns (coroutines may own system resources and do cleanup in `finally` blocks, and if you don’t await a coroutine then you’ll never reach the `finally` block and so will leak system resources). Given that there’s no computational way to be sure that *not* running a coroutine is safe (hello there halting problem), asyncio takes the pedantic model and says that not running a coroutine is a condition that justifies a warning. I think asyncio’s position here is correct, incidentally.
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern?
Yes. This is the simple map to your old API, and is absolutely what I’d recommend doing in the first instance if you want to use coroutines.
- Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
There are a few other patterns you could use. The first is to return a Future, and just always run the “polling” function in the background to resolve that future. If the caller doesn’t care about the result they can just ignore the Future, and if they do care they can await on it. This has the downside of always requiring the I/O to poll, but is otherwise pretty clean. Another option is to offer two functions, for example `def resource_nowait()` and `async def resource`. The caller can decide which they want to call based on whether they care about finding out the result. This is the clearest approach that doesn’t trigger automatic extra work like the Future does, and it lacks magic: it’s very declarative. This is a nice approach for keeping things clean. Finally, you can create a magic awaitable object. PEP 492 defines several ways to create an “awaitable”, but one approach is to use what it calls a “Future-like object”: that is, one with a __await__ method that returns an iterator. In this case, you’d do a very basic extension of the Future object by triggering the work upon the call of __await__ before delegating to the normal behaviour. This is an annoyingly precise thing to do, though technically do-able. Cory
On Wed, Jul 12, 2017 at 9:44 AM, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
@property def future(self): if self._future is None: self._future = asyncio.Future() asyncio.ensure_future(self._poll()) return self._future
result = await poller.future
If I don't call the "future" attribute, I don't poll at all. My initial "create" method returns the same object anytime, and I don't need a parameter or another "nowait" method. Do you have any caveat for issues I don’t see in this approach?
Hi Laurent, it seems like there is still an issue with this approach in that merely accessing / inspecting the property has the side effect of creating a coroutine object (calling self._poll()), and so can trigger the warning in innocent-looking code: loop = asyncio.get_event_loop() poller = PollerOperation() fut = poller.future # creates coroutine object if False: loop.run_until_complete(fut) loop.close() I'm not sure if others feel differently, but property access IMO shouldn't have possible side effects like this. If there are possible negative side effects, it should be a method call to indicate to the user that it is "doing" something that warrants more consideration. --Chris
Thank you very much!!!
Laurent
PS: Cory, I just subscribed to the mailing list 😊
-----Original Message----- From: Cory Benfield [mailto:cory@lukasa.co.uk] Sent: Wednesday, July 12, 2017 01:18 To: Laurent Mazuel <lmazuel@microsoft.com> Cc: async-sig@python.org Subject: Re: [Async-sig] Optional async method and best practices
On 11 Jul 2017, at 23:26, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
Hi Laurent! A future note: your message got stuck in moderation because you aren’t subscribed to the mailing list. You may find it helpful to subscribe, as your future messages will also get stuck unless you do!
My first prototype was to split and return a tuple (resource, coroutine):
obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited
I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.
The assumption in asyncio, generally speaking, is that you do not create coroutines you do not care about running. This is both for abstract theoretical reasons (if you don’t care if the coroutine is run or not, why not just optimise your code to never create the coroutine and save yourself the CPU cycles?) and for more concrete practical concerns (coroutines may own system resources and do cleanup in `finally` blocks, and if you don’t await a coroutine then you’ll never reach the `finally` block and so will leak system resources).
Given that there’s no computational way to be sure that *not* running a coroutine is safe (hello there halting problem), asyncio takes the pedantic model and says that not running a coroutine is a condition that justifies a warning. I think asyncio’s position here is correct, incidentally.
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern?
Yes. This is the simple map to your old API, and is absolutely what I’d recommend doing in the first instance if you want to use coroutines.
- Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
There are a few other patterns you could use.
The first is to return a Future, and just always run the “polling” function in the background to resolve that future. If the caller doesn’t care about the result they can just ignore the Future, and if they do care they can await on it. This has the downside of always requiring the I/O to poll, but is otherwise pretty clean.
Another option is to offer two functions, for example `def resource_nowait()` and `async def resource`. The caller can decide which they want to call based on whether they care about finding out the result. This is the clearest approach that doesn’t trigger automatic extra work like the Future does, and it lacks magic: it’s very declarative. This is a nice approach for keeping things clean.
Finally, you can create a magic awaitable object. PEP 492 defines several ways to create an “awaitable”, but one approach is to use what it calls a “Future-like object”: that is, one with a __await__ method that returns an iterator. In this case, you’d do a very basic extension of the Future object by triggering the work upon the call of __await__ before delegating to the normal behaviour. This is an annoyingly precise thing to do, though technically do-able.
Cory _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
Good point! I didn't see it. It will definitely happen. So I won't do that 😊 So, my best candidate right now is the "get_resource(nowait)" two methods approach. Thank you! Laurent -----Original Message----- From: Chris Jerdonek [mailto:chris.jerdonek@gmail.com] Sent: Wednesday, July 12, 2017 13:27 To: Laurent Mazuel <lmazuel@microsoft.com> Cc: Cory Benfield <cory@lukasa.co.uk>; async-sig@python.org Subject: Re: [Async-sig] Optional async method and best practices On Wed, Jul 12, 2017 at 9:44 AM, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
@property def future(self): if self._future is None: self._future = asyncio.Future() asyncio.ensure_future(self._poll()) return self._future
result = await poller.future
If I don't call the "future" attribute, I don't poll at all. My initial "create" method returns the same object anytime, and I don't need a parameter or another "nowait" method. Do you have any caveat for issues I don’t see in this approach?
Hi Laurent, it seems like there is still an issue with this approach in that merely accessing / inspecting the property has the side effect of creating a coroutine object (calling self._poll()), and so can trigger the warning in innocent-looking code: loop = asyncio.get_event_loop() poller = PollerOperation() fut = poller.future # creates coroutine object if False: loop.run_until_complete(fut) loop.close() I'm not sure if others feel differently, but property access IMO shouldn't have possible side effects like this. If there are possible negative side effects, it should be a method call to indicate to the user that it is "doing" something that warrants more consideration. --Chris
Thank you very much!!!
Laurent
PS: Cory, I just subscribed to the mailing list 😊
-----Original Message----- From: Cory Benfield [mailto:cory@lukasa.co.uk] Sent: Wednesday, July 12, 2017 01:18 To: Laurent Mazuel <lmazuel@microsoft.com> Cc: async-sig@python.org Subject: Re: [Async-sig] Optional async method and best practices
On 11 Jul 2017, at 23:26, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
Hi Laurent! A future note: your message got stuck in moderation because you aren’t subscribed to the mailing list. You may find it helpful to subscribe, as your future messages will also get stuck unless you do!
My first prototype was to split and return a tuple (resource, coroutine):
obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited
I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.
The assumption in asyncio, generally speaking, is that you do not create coroutines you do not care about running. This is both for abstract theoretical reasons (if you don’t care if the coroutine is run or not, why not just optimise your code to never create the coroutine and save yourself the CPU cycles?) and for more concrete practical concerns (coroutines may own system resources and do cleanup in `finally` blocks, and if you don’t await a coroutine then you’ll never reach the `finally` block and so will leak system resources).
Given that there’s no computational way to be sure that *not* running a coroutine is safe (hello there halting problem), asyncio takes the pedantic model and says that not running a coroutine is a condition that justifies a warning. I think asyncio’s position here is correct, incidentally.
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern?
Yes. This is the simple map to your old API, and is absolutely what I’d recommend doing in the first instance if you want to use coroutines.
- Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
There are a few other patterns you could use.
The first is to return a Future, and just always run the “polling” function in the background to resolve that future. If the caller doesn’t care about the result they can just ignore the Future, and if they do care they can await on it. This has the downside of always requiring the I/O to poll, but is otherwise pretty clean.
Another option is to offer two functions, for example `def resource_nowait()` and `async def resource`. The caller can decide which they want to call based on whether they care about finding out the result. This is the clearest approach that doesn’t trigger automatic extra work like the Future does, and it lacks magic: it’s very declarative. This is a nice approach for keeping things clean.
Finally, you can create a magic awaitable object. PEP 492 defines several ways to create an “awaitable”, but one approach is to use what it calls a “Future-like object”: that is, one with a __await__ method that returns an iterator. In this case, you’d do a very basic extension of the Future object by triggering the work upon the call of __await__ before delegating to the normal behaviour. This is an annoyingly precise thing to do, though technically do-able.
Cory _______________________________________________ Async-sig mailing list Async-sig@python.org https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail. python.org%2Fmailman%2Flistinfo%2Fasync-sig&data=02%7C01%7Clmazuel%40m icrosoft.com%7Cc1013246fdbf4f56115808d4c964635a%7C72f988bf86f141af91ab 2d7cd011db47%7C1%7C0%7C636354880389311043&sdata=js9Q%2BwiV6IgMDgLDQs7e shN8N0QCZypP7qX0IanKOlU%3D&reserved=0 Code of Conduct: https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.p ython.org%2Fpsf%2Fcodeofconduct%2F&data=02%7C01%7Clmazuel%40microsoft. com%7Cc1013246fdbf4f56115808d4c964635a%7C72f988bf86f141af91ab2d7cd011d b47%7C1%7C0%7C636354880389311043&sdata=gmlfMEf3RQ6GXW4WwrPuPlTWWdxAqNa muq8W%2B01LR6M%3D&reserved=0
Hi Laurent, I'm still a dilettante, so take my comments with a grain of salt: 1. Target Python 3.6 only. (i.e. drop 3.5; look at 3.7 obv, but you want users now) (i.e. forget `yield from`, none will remember/get it next year) (if 2.7 or 3.3 must be supported, provide synch package) 2. Use futures (unless it's a stream) 3. Shield liberally 4. Provide context managers Naive user code might look like that: req = lib.request(...) await req.ready() # optional return await req.json()["something"] That's sane and pretty similar to http://aiohttp.readthedocs.io/en/stable/client.html thus your users will get it :) A more advanced use will be `[async] with lib.request(...) as r: await r.json()` (you probably want `async with` unless you can ensure synchronous timely termination) Personally I'd prefer `ready` and `json` without parentheses, but it seems I'm in a minority. Cheers, d. On 12 July 2017 at 00:26, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
I’m working currently with Brett Cannon to bring asyncio support to our SDK. We wanted to check with you one of the scenario, since we got a loooong discussion on it together 😊. And we want to do it using the best reasonable practice with your opinion.
We have an api that is clearly async and will gain a lot to be converted to asyncio. However, it's a two-step operation. Operation 1 asks for the creation of a resource and is not async, operation 2 is *optional* and wait for completion of this creation (with nightmare threads currently and I removed a lot of code moving to asyncio - happiness). There is perfectly legit scenarios where operation 2 is not needed and avoid it is better, but it has to be prepared at the same time of operation 1. Current code looks like this:
sync_poller = client.create(**parameters) obj = sync_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = sync_poller.result() # OPTIONAL. This is a blocking call with thread, if you want to wait for actual creation and get updated metadatas
My first prototype was to split and return a tuple (resource, coroutine):
obj, optional_poller = client.create(**parameters) obj = await optional_poller # OPTIONAL
But I got a warning if I decide to do not use this poller, RuntimeWarning: coroutine 'foo' was never awaited
I was surprised honestly that I can't do that, since I feel like I'm not leaking anything. I didn't run the operation, so there is no wasted resource at my knowledge. But I remember wasting time because of a forgotten "yield from", so I guess it's fair 😊. But I would be curious to understand what I did badly.
I found 2 solutions to avoid the warning, and I currently prefer solution 2: 1- Return a function to call, and not a coroutine. The "await" statement becomes:
obj = await optional_poller()
2- Return my initial object with an async method. This allows me to write (something finally close to the current code):
async_poller = client.create(**parameters) obj = async_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = await async_poller.result() # OPTIONAL
My async_poller object being something like:
class PollerOperation: async def result(self): ...async version of previous sync result()...
So the questions are: - Does this seem a correct pattern? - Is there a simple way to achieve something like this:
obj = await async_poller
meaning, I can win the "result()" syntax and directly "await" on the object and get the result from magic function. I tried by subclassing some ABC coroutine/awaitable, but wasn't able to find a correct syntax. I'm not even sure this makes sense and respects the zen of Python 😊
If it helps, I'm willing to use 3.5 as minimal requirement to get async behavior.
Thank you!!
Laurent _______________________________________________ Async-sig mailing list Async-sig@python.org https://mail.python.org/mailman/listinfo/async-sig Code of Conduct: https://www.python.org/psf/codeofconduct/
On Tue, Jul 11, 2017 at 3:26 PM, Laurent Mazuel via Async-sig <async-sig@python.org> wrote:
Hello,
I’m working currently with Brett Cannon to bring asyncio support to our SDK. We wanted to check with you one of the scenario, since we got a loooong discussion on it together 😊. And we want to do it using the best reasonable practice with your opinion.
We have an api that is clearly async and will gain a lot to be converted to asyncio. However, it's a two-step operation. Operation 1 asks for the creation of a resource and is not async, operation 2 is *optional* and wait for completion of this creation (with nightmare threads currently and I removed a lot of code moving to asyncio - happiness). There is perfectly legit scenarios where operation 2 is not needed and avoid it is better, but it has to be prepared at the same time of operation 1. Current code looks like this:
sync_poller = client.create(**parameters) obj = sync_poller.resource() # Get the initial resource information, but the object is not actually created yet. obj = sync_poller.result() # OPTIONAL. This is a blocking call with thread, if you want to wait for actual creation and get updated metadatas
My advice would be to - think how you'd write the API if it were synchronous, and then do exactly that, except marking the blocking functions/methods as async - pretend that the only allowed syntax for using await or coroutines is 'await fn(...)', and treat the 'await obj' syntax as something that only exists for compatibility with legacy code that uses explicit Future/Deferred objects. One of Python's famous design principles is "there should be one (and preferably only one) obvious way to do it". The word "obvious" there is important -- because programming is so flexible, and Python in particular is so flexible, there generally are tons and tons of ways to do anything in Python. Like, if you want to make an http request, that could be a function call, `requests.get(url)`. Or you could have a class that does the request when you access some property, `Request(url).doit`. Or you could have a class whose `__str__` method does the request, and you monkeypatch a StringIO in place of sys.stdout and use print, like `sys.stdout = io.StringIO(); print(Request(url), end=""); body = sys.stdout.getvalue()`. Why not, the computer doesn't care! These are all equally compliant with the Python language specification. Fortunately, we have some strong conventions about these things, and most of these options would never even occur to most people. *Of course* the obvious way to make an HTTP request is to call a function. The other options are ridiculous. And that makes life much easier, because we don't need to stop every time we implement some trivial function and be like "hmm, what if I did this using monkeypatching and two metaclasses? Would that be a good idea?", and it means that when you use a new library you (mostly) don't have to worry that they're secretly using monkeypatching and metaclasses to implement some trivial functionality, etc. Yay conventions. But *un*fortunately, async in Python is so new and shiny that we currently have all this flexibility, but we don't have conventions yet, so people try all kinds of wacky stuff and no-one's sure what to recommend. There's lots of ways to do it, but no-one knows which one is obvious. The nice thing about my rules above is that they give you one obvious way to do it, and minimize the delta between sync and async code. You already know how functions and synchronous APIs work, your users already know how functions and synchronous APIs work, all you need to add is some 'async' and 'await' keywords and you're good to go. I think forcing users to know what "coroutine objects" are before they can write async I/O code – or even deal with the word "coroutine" at all – is like forcing users to understand the nuances of metaclass property lookup and the historical mess around nb_add/sq_concat before they can use a+b to add two numbers. Obviously in both cases it's important that the full details are available for those who want to dig into it, but you shouldn't need that just to make some HTTP requests. tl;dr: Treat async functions as a kind of function with a funny calling convention. -n -- Nathaniel J. Smith -- https://vorpus.org
participants (5)
-
Chris Jerdonek -
Cory Benfield -
Dima Tisnek -
Laurent Mazuel -
Nathaniel Smith