[Python-ideas] Subprocess: Add an encoding argument

[Paul Moore]

On 2 September 2014 14:25, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 2 September 2014 07:15, Paul Moore <p.f.moore at gmail.com> wrote:
>> The find -print0 example is out of scope, IMO, as newline handling is
>> different from encoding. At some point, it becomes easier to manually
>> wrap the streams rather than having huge numbers of parameters to the
>> Popen constructor.
>
> Don't forget Antoine's suggestion of creating a TextPopen subclass
> that wraps the streams as strict UTF-8 by default and allows the
> encoding and errors arguments to be either strings (affecting all
> pipes) or a dictionary mapping "stdin", "stdout" and "stderr" to
> individual settings.
>
> With that, the simple utf-8 example just becomes:
>
>   with TextPopen(cmd, stdout=PIPE) as p:
>       for line in p.stdout:
>           process(line)

I'd not forgotten that, but it doesn't help for the -print0 case,
which is about using nul as a line ending, and not about encodings.
I'm going to carefully avoid getting sucked into that open issue here,
and stick to only considering encodings :-)

>> I'll think some more on this...
>
> For your torture test, consider the "iconv" (or "win_iconv") utility,
> which does encoding conversions, and how you might test that from a
> Python program without needing to do your own encoding and decoding,
> but instead let the subprocess module handle it for you :)

That's another good use case for this functionality.
Paul