Allow creation of polymorph function (async function executable syncronously)
I was doing some async networking and I wondered, why I have to use 2 different api for making the same things in async or sync regime. Even if we make 2 perfectly identical api (except function will be sync and async), it will still 2 different code). So I first thinked to allow await in syncronous function but that create some problems (ex: an async function calling async.create_task) so if we allow that we have to asume to allways be in async regime (like js). Or we can differentiate async function wich can be awaited in syncronous regime, maybe with a new keyword (here I will use polymorph due to a lack of imagination but I find that one too long) ? So a polymorph function can be awaited in a syncronous function, and a polymorph function can only await polymorph functions. Polymorph function work exacly like async function BUT they assure of the ability to execute syncronously. And in a syncronous regime if an await need to wait (like async.sleep or network operation), just wait (like the equivalent of this function in syncronous way). So why made that ? To provide the same api for async and sync regime when its possible, example http api. This allow to code less librairy. Syncronous users can just use the librairy like any other sync lib (with the keyword await for executing but, personally, I think that is worth). And asyncronous users can run multiples tasks using the same lib. Moving from a regime to an other is simpler, source code size is reduced (doesn't need to create 2 api for the same lib), gain of time for the same reason. Also why it need to await in syncronous function, why not just execute polymorph function like any sync function while called in a sync function ? Because we need to create runnable objects for async.run, ... So I would have your though about that, what can be improved, a better name for polymorph ?
Defining a single polymorphic function is easy at the library level. For example, with asyncio: ---- def maybe_async(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): coro = fn(*args, **kwargs) if asyncio.get_running_loop() is not None: return coro else: return await coro @maybe_async async def my_func(...): ... use asyncio freely in here ... ---- You can't do it at the language level though (e.g. with your proposed 'polymorph' keyword), because the language doesn't know whether an event loop is running or not. Extending this from a single function to a whole library API is substantially more complex, because you have to wrap every function and method, deal with __iter__ versus __aiter__, etc. -n On Tue, Mar 5, 2019 at 8:02 PM Jorropo . <jorropo.pgm@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
Is this what syncer does with a sync() function? Why isn't this a built-in? https://github.com/miyakogi/syncer On Wednesday, March 6, 2019, Nathaniel Smith <njs@pobox.com> wrote:
Jorropo states: Polymorph function work exacly like async function BUT they assure of the
ability to execute syncronously. - sic
Async functions can call sync functions, but not vice versa. Consider a third party solution - trio <https://github.com/python-trio/trio>, that allows sync functions to call async functions.
From the docs <https://trio.readthedocs.io/en/latest/tutorial.html#async-functions>:
async def async_double(x): return 2 * x trio.run(async_double, 3) # returns 6 --- If you want a stdlib solution, let's revisit Nathaniel Smith's example:
I was unable to run his example as-is (in Python 3.6 at least) since the `await` keyword is only permitted inside an `async def` function. However, the idea is intriguing and can be adapted. See the example below. Code: def maybe_async(fn): async def _process(fn, *args, **kwargs): coro_fn = fn(*args, **kwargs) if asyncio.iscoroutinefunction(fn): return await coro_fn else: return coro_fn @functools.wraps(fn) def wrapper(*args, **kwarg): loop = asyncio.get_event_loop() res = loop.run_until_complete(_process(fn, *args, **kwarg)) return res return wrapper Demo: @maybe_async async def agreet(delay): print("hello") await asyncio.sleep(delay) print("world") @maybe_async def greet(delay): print("hello") time.sleep(delay) print("world") agreet(2) # prints hello world after 2 seconds greet(1) # print hello world after 1 second Now you can call either sync or async functions like regular functions. Hope this helps. --- On Wed, Mar 6, 2019 at 12:54 AM Nathaniel Smith <njs@pobox.com> wrote:
On Wed, Mar 6, 2019 at 4:37 PM pylang <pylang3@gmail.com> wrote:
Oh yeah, that was a brain fart. I meant to write: def maybe_async(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): coro = fn(*args, **kwargs) if asyncio.get_running_loop() is not None: return coro else: return asyncio.run(coro) -n -- Nathaniel J. Smith -- https://vorpus.org
Here's syncer/syncer.py: https://github.com/miyakogi/syncer/blob/master/syncer.py I think the singledispatch is pretty cool. ```python #!/usr/bin/env python3 # -*- coding: utf-8 -*- import sys from functools import singledispatch, wraps import asyncio import inspect import types from typing import Any, Callable, Generator PY35 = sys.version_info >= (3, 5) def _is_awaitable(co: Generator[Any, None, Any]) -> bool: if PY35: return inspect.isawaitable(co) else: return (isinstance(co, types.GeneratorType) or isinstance(co, asyncio.Future)) @singledispatch def sync(co: Any): raise TypeError('Called with unsupported argument: {}'.format(co)) @sync.register(asyncio.Future) @sync.register(types.GeneratorType) def sync_co(co: Generator[Any, None, Any]) -> Any: if not _is_awaitable(co): raise TypeError('Called with unsupported argument: {}'.format(co)) return asyncio.get_event_loop().run_until_complete(co) @sync.register(types.FunctionType) @sync.register(types.MethodType) def sync_fu(f: Callable[..., Any]) -> Callable[..., Any]: if not asyncio.iscoroutinefunction(f): raise TypeError('Called with unsupported argument: {}'.format(f)) @wraps(f) def run(*args, **kwargs): return asyncio.get_event_loop().run_until_complete(f(*args, **kwargs)) return run if PY35: sync.register(types.CoroutineType)(sync_co) ``` On Wed, Mar 6, 2019 at 9:20 PM Nathaniel Smith <njs@pobox.com> wrote:
Defining a single polymorphic function is easy at the library level. For example, with asyncio: ---- def maybe_async(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): coro = fn(*args, **kwargs) if asyncio.get_running_loop() is not None: return coro else: return await coro @maybe_async async def my_func(...): ... use asyncio freely in here ... ---- You can't do it at the language level though (e.g. with your proposed 'polymorph' keyword), because the language doesn't know whether an event loop is running or not. Extending this from a single function to a whole library API is substantially more complex, because you have to wrap every function and method, deal with __iter__ versus __aiter__, etc. -n On Tue, Mar 5, 2019 at 8:02 PM Jorropo . <jorropo.pgm@gmail.com> wrote:
-- Nathaniel J. Smith -- https://vorpus.org
Is this what syncer does with a sync() function? Why isn't this a built-in? https://github.com/miyakogi/syncer On Wednesday, March 6, 2019, Nathaniel Smith <njs@pobox.com> wrote:
Jorropo states: Polymorph function work exacly like async function BUT they assure of the
ability to execute syncronously. - sic
Async functions can call sync functions, but not vice versa. Consider a third party solution - trio <https://github.com/python-trio/trio>, that allows sync functions to call async functions.
From the docs <https://trio.readthedocs.io/en/latest/tutorial.html#async-functions>:
async def async_double(x): return 2 * x trio.run(async_double, 3) # returns 6 --- If you want a stdlib solution, let's revisit Nathaniel Smith's example:
I was unable to run his example as-is (in Python 3.6 at least) since the `await` keyword is only permitted inside an `async def` function. However, the idea is intriguing and can be adapted. See the example below. Code: def maybe_async(fn): async def _process(fn, *args, **kwargs): coro_fn = fn(*args, **kwargs) if asyncio.iscoroutinefunction(fn): return await coro_fn else: return coro_fn @functools.wraps(fn) def wrapper(*args, **kwarg): loop = asyncio.get_event_loop() res = loop.run_until_complete(_process(fn, *args, **kwarg)) return res return wrapper Demo: @maybe_async async def agreet(delay): print("hello") await asyncio.sleep(delay) print("world") @maybe_async def greet(delay): print("hello") time.sleep(delay) print("world") agreet(2) # prints hello world after 2 seconds greet(1) # print hello world after 1 second Now you can call either sync or async functions like regular functions. Hope this helps. --- On Wed, Mar 6, 2019 at 12:54 AM Nathaniel Smith <njs@pobox.com> wrote:
On Wed, Mar 6, 2019 at 4:37 PM pylang <pylang3@gmail.com> wrote:
Oh yeah, that was a brain fart. I meant to write: def maybe_async(fn): @functools.wraps(fn) def wrapper(*args, **kwargs): coro = fn(*args, **kwargs) if asyncio.get_running_loop() is not None: return coro else: return asyncio.run(coro) -n -- Nathaniel J. Smith -- https://vorpus.org
Here's syncer/syncer.py: https://github.com/miyakogi/syncer/blob/master/syncer.py I think the singledispatch is pretty cool. ```python #!/usr/bin/env python3 # -*- coding: utf-8 -*- import sys from functools import singledispatch, wraps import asyncio import inspect import types from typing import Any, Callable, Generator PY35 = sys.version_info >= (3, 5) def _is_awaitable(co: Generator[Any, None, Any]) -> bool: if PY35: return inspect.isawaitable(co) else: return (isinstance(co, types.GeneratorType) or isinstance(co, asyncio.Future)) @singledispatch def sync(co: Any): raise TypeError('Called with unsupported argument: {}'.format(co)) @sync.register(asyncio.Future) @sync.register(types.GeneratorType) def sync_co(co: Generator[Any, None, Any]) -> Any: if not _is_awaitable(co): raise TypeError('Called with unsupported argument: {}'.format(co)) return asyncio.get_event_loop().run_until_complete(co) @sync.register(types.FunctionType) @sync.register(types.MethodType) def sync_fu(f: Callable[..., Any]) -> Callable[..., Any]: if not asyncio.iscoroutinefunction(f): raise TypeError('Called with unsupported argument: {}'.format(f)) @wraps(f) def run(*args, **kwargs): return asyncio.get_event_loop().run_until_complete(f(*args, **kwargs)) return run if PY35: sync.register(types.CoroutineType)(sync_co) ``` On Wed, Mar 6, 2019 at 9:20 PM Nathaniel Smith <njs@pobox.com> wrote:
participants (4)
-
Jorropo . -
Nathaniel Smith -
pylang -
Wes Turner