[Async-sig] async testing question

[Dima Tisnek]

I'd say mock 2nd to `await time.sleep(1); assert False, "should not
happen"` with the earlier just in case test harness or code under test
is broken.

The tricky part is how to cancel your library function at the right
time (i.e. not too early).

You could, perhaps, mock 1st call to
`ensure_future(async_cancel_task())` but imagine that code under test
gets changed to:

async to_be_tested():
  await first()
  logging.debug("...")  # you don't expect event loop interaction
here, but what if?
  await second()
  await third()

If it's all right for your test to fail on such a change, then fine :)
If you consider that unexpected breakage, then I dunno what you can do :P

On 1 July 2017 at 22:06, Chris Jerdonek <chris.jerdonek at gmail.com> wrote:
> On Sat, Jul 1, 2017 at 3:49 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
>> Hi Chris,
>>
>> This specific test is easy to write (mock first to return a resolved future,
>> 2nd to block and 3rd to assert False)
>
> Saying it's easy doesn't necessarily help the questioner. :)
>
> Issues around combinatorics I understand. It's more the mechanics of
> the basic testing pattern I'd like advice on.
>
> For example, if I mock the second function to be blocking, how do I
> invoke the higher-level function in a way so I can continue at the
> point where the second function blocks? And without introducing
> brittleness or relying on implementation details of the event loop?
>
> (By the way, it seems you wouldn't want to mock the third function in
> cases like if the proper handling of task.cancel() depends on the
> behavior of the third function, for example if CancelledError is being
> caught.)
>
> --Chris
>
>>
>> OTOH complexity of the general case is unbounded and generally exponential.
>> It's akin to testing multithreaded code.
>> (There's an academic publication from Microsoft where they built a runtime
>> that would run each test really many times, where scheduler is rigged to
>> order runnable tasks differently on each run. I hope someone rewrites this
>> for asyncio)
>>
>> Certainty [better] tools are needed, and ultimately it's a tradeoff between
>> sane/understable/maintainable tests and testing deeper/more corner cases.
>>
>> Just my 2c...
>>
>> On Jul 1, 2017 12:11, "Chris Jerdonek" <chris.jerdonek at gmail.com> wrote:
>>>
>>> I have a question about testing async code.
>>>
>>> Say I have a coroutine:
>>>
>>>     async def do_things():
>>>         await do_something()
>>>         await do_more()
>>>         await do_even_more()
>>>
>>> And future:
>>>
>>>     task = ensure_future(do_things())
>>>
>>> Is there a way to write a test case to check that task.cancel() would
>>> behave correctly if, say, do_things() is waiting at the line
>>> do_more()?
>>>
>>> In real life, this situation can happen if a function like the
>>> following is called, and an exception happens in one of the given
>>> tasks.  One of the tasks in the "pending" list could be at the line
>>> do_more().
>>>
>>>     done, pending = await asyncio.wait(tasks,
>>>                          return_when=asyncio.FIRST_EXCEPTION)
>>>
>>> But in a testing situation, you don't necessarily have control over
>>> where each task ends up when FIRST_EXCEPTION occurs.
>>>
>>> Thanks,
>>> --Chris
>>> _______________________________________________
>>> Async-sig mailing list
>>> Async-sig at python.org
>>> https://mail.python.org/mailman/listinfo/async-sig
>>> Code of Conduct: https://www.python.org/psf/codeofconduct/