[New-bugs-announce] [issue45109] pipes seems designed for bytes but is str-only

[Richard Tollerton]

New submission from Richard Tollerton <rich.tollerton at ni.com>:

1. https://github.com/python/cpython/blob/3.9/Lib/pipes.py#L6

> Suppose you have some data that you want to convert to another format,
> such as from GIF image format to PPM image format.

2. https://docs.python.org/3.9/library/pipes.html

> Because the module uses /bin/sh command lines, a POSIX or compatible shell for os.system() and os.popen() is required.

3. https://docs.python.org/3.9/library/os.html#os.popen

> The returned file object reads or writes text strings rather than bytes.


(1) and (3) are AFAIK mutually contradictory: you can't reasonably expect to shove GIFs down a str file object. I'm guessing that pipes is an API that never got its bytes API fleshed out?

My main interest in this is that I'm writing a large CSV to disk and wanted to pipe it through zstd first. And I wanted something like perl's open FILE, "|zstd -T0 -19 > out.txt.zst". But the CSV at present is all bytes. (Technically the content is all latin1 at the moment, so I may have a workaround, but I'm not 100% certain it will stay that way.)

What I'd like to see is for pipes.Template.open() to accept 'b' in flags, and for that to be handled in the usual way.

----------
components: Library (Lib)
messages: 401103
nosy: rtollert
priority: normal
severity: normal
status: open
title: pipes seems designed for bytes but is str-only
versions: Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45109>
_______________________________________