[New-bugs-announce] [issue31087] asyncio.create_subprocess_* do not honor `encoding`

[Alex]

New submission from Alex:

Regardless of the value of `encoding`, StreamReaders returned for the `asyncio.subprocess.Process`'s `stdout` and `stderr` would be in binary mode.

    import sys
    import asyncio
    import subprocess

    async def main():
        sp = await asyncio.create_subprocess_exec('ls', '-la',
                                                  stdin=None,
                                                  stdout=subprocess.PIPE,
                                                  encoding='utf8')
        await sp.wait()
        data = await sp.stdout.read()
        print(data)
        print(isinstance(data, bytes))

    loop = asyncio.get_event_loop()
    loop.run_until_complete(main())

There are two naive solutions:

- `create_subprocess_*` could explicitly raise when provided any `encoding` and `errors` (like it now does with `universal_newlines`).

- or implement encoding conversions for StreamReaders (and StreamWriters), and forward those.

Of course it would be better to have conversions, but I don't know how those will have to deal with decoding partially available data - e.g. returning in the middle of surrogate pair, or having only half a codepoint for UTF16. There should likely cache partial data to process at the next read, but this would require rewriting codecs to support partial decode... seems like it's not that easy :(

----------
messages: 299537
nosy: toriningen
priority: normal
severity: normal
status: open
title: asyncio.create_subprocess_* do not honor `encoding`

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue31087>
_______________________________________