[Async-sig] async testing question

[Chris Jerdonek]

On Sat, Jul 1, 2017 at 3:49 AM, Dima Tisnek <dimaqq at gmail.com> wrote:
> Hi Chris,
>
> This specific test is easy to write (mock first to return a resolved future,
> 2nd to block and 3rd to assert False)

Saying it's easy doesn't necessarily help the questioner. :)

Issues around combinatorics I understand. It's more the mechanics of
the basic testing pattern I'd like advice on.

For example, if I mock the second function to be blocking, how do I
invoke the higher-level function in a way so I can continue at the
point where the second function blocks? And without introducing
brittleness or relying on implementation details of the event loop?

(By the way, it seems you wouldn't want to mock the third function in
cases like if the proper handling of task.cancel() depends on the
behavior of the third function, for example if CancelledError is being
caught.)

--Chris

>
> OTOH complexity of the general case is unbounded and generally exponential.
> It's akin to testing multithreaded code.
> (There's an academic publication from Microsoft where they built a runtime
> that would run each test really many times, where scheduler is rigged to
> order runnable tasks differently on each run. I hope someone rewrites this
> for asyncio)
>
> Certainty [better] tools are needed, and ultimately it's a tradeoff between
> sane/understable/maintainable tests and testing deeper/more corner cases.
>
> Just my 2c...
>
> On Jul 1, 2017 12:11, "Chris Jerdonek" <chris.jerdonek at gmail.com> wrote:
>>
>> I have a question about testing async code.
>>
>> Say I have a coroutine:
>>
>>     async def do_things():
>>         await do_something()
>>         await do_more()
>>         await do_even_more()
>>
>> And future:
>>
>>     task = ensure_future(do_things())
>>
>> Is there a way to write a test case to check that task.cancel() would
>> behave correctly if, say, do_things() is waiting at the line
>> do_more()?
>>
>> In real life, this situation can happen if a function like the
>> following is called, and an exception happens in one of the given
>> tasks.  One of the tasks in the "pending" list could be at the line
>> do_more().
>>
>>     done, pending = await asyncio.wait(tasks,
>>                          return_when=asyncio.FIRST_EXCEPTION)
>>
>> But in a testing situation, you don't necessarily have control over
>> where each task ends up when FIRST_EXCEPTION occurs.
>>
>> Thanks,
>> --Chris
>> _______________________________________________
>> Async-sig mailing list
>> Async-sig at python.org
>> https://mail.python.org/mailman/listinfo/async-sig
>> Code of Conduct: https://www.python.org/psf/codeofconduct/