[New-bugs-announce] [issue23812] asyncio.Queue.put_nowait(), followed get() task cancellation leads to item being lost
Gustavo J. A. M. Carneiro
report at bugs.python.org
Mon Mar 30 15:59:52 CEST 2015
New submission from Gustavo J. A. M. Carneiro:
I have a pattern where I read from a queue with a timeout, generally like this:
reader = asyncio.async(wait_for(queue.get(), 0.1))
item = (yield from reader)
This is to have a loop where we try to get items from the queue with a timeout. Prior to Python 3.5, wait_for() doesn't automatically cancel the reading task, so I have to do it explicitly above.
The code has a strange race condition where, if a taks calls put_nowait on a reader task that is just about to be cancelled due to timeout, then the item that wast put onto the queue gets lost.
In the tests framework, the minimal test case I can come up with is mainly this code:
q = asyncio.Queue(loop=loop)
reader = loop.create_task(q.get())
When the reader gets cancelled, the item `1` is lost into the ether, and when you create another reader for the same queue, it only gets the second item `2`. I would expect that, if a reader task is cancelled, the item at the head of the queue doesn't get lost. Either the reader succeeds and the item is removed, or it doesn't succeed and the item is left alone.
I attach a patch to the tulip tests that reproduces the problem in both Python 3.4.2 and tulip hg.
nosy: gustavo, gvanrossum, haypo, yselivanov
title: asyncio.Queue.put_nowait(), followed get() task cancellation leads to item being lost
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file38740/bug-test.diff
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce