[New-bugs-announce] [issue32395] asyncio.StreamReader.readuntil is not general enough

[Bruce Merry]

New submission from Bruce Merry <bmerry at gmail.com>:

I'd proposed one specific solution in Issue 32052 which asvetlov didn't like, so as requested I'm filing a bug about the problem rather than the solution.

The specific case I have is reading a protocol in which either \r or \n can be used to terminate lines. With StreamReader.readuntil, it's only possible to specify one separator, so it can't easily be used (*).

Some nice-to-have features, from specific to general:
1. Specify multiple alternate separators.
2. Specify a regex for a separator.
3. Specify a regex for the line.
4. Specify a callback that takes a string and returns the position of the end of the line, if any.

Of course, some of these risk quadratic-time behaviour if they have to check the whole buffer every time the buffer is extended, so that would need to be considered in the design. In the last case, the callback could take care of it itself by maintaining internal state.


(*) I actually have a solution for this case (https://github.com/ska-sa/aiokatcp/blob/bd8263cefe213003a218fac0dd8c5207cc76aeef/aiokatcp/connection.py#L44-L52), but it only works because \r and \n are semantically equivalent in the particular protocol I'm parsing.

----------
components: asyncio
messages: 308852
nosy: Bruce Merry, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.StreamReader.readuntil is not general enough
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32395>
_______________________________________