[Python-ideas] struct.unpack should support open files

Chris Angelico rosuav at gmail.com
Wed Dec 26 21:15:46 EST 2018

I'm quoting Steve's post here but am responding more broadly to the
whole thread too.

On Thu, Dec 27, 2018 at 1:00 PM Steven D'Aprano <steve at pearwood.info> wrote:
> I assume you have no objection to the existence of json.load() and
> json.loads() functions. (If you do think they're a bad idea, I don't
> know what to say.) Have they lead to "an open ended can of worms"?

Personally, I'd actually be -0 on json.load if it didn't already
exist. It's just a thin wrapper around json.loads() - it doesn't
actually add anything. This proposal is _notably better_ in that it
will (attempt to) read the correct number of bytes. The only real
reason to have json.load/json.loads is to match pickle etc. (Though
pickle does things the other way around, at least in the Python source
code I have handy - loads is implemented using BytesIO, so it's the
file-based API that's fundamental, as opposed to JSON where the
string-based API is fundamental. I guess maybe that's a valid reason?
To allow either one to be implemented in terms of the other?) But
reading a struct *and then leaving the rest behind* is, IMO, a more
valuable feature.

> Let's save backwards compatibility:
> - unpack() reads from strings
> - unpackf() reads from files
> Does this open a can of worms?

Not in my opinion, but I also don't think it gains you anything much.
It isn't consistent with other stdlib modules, and it isn't very
advantageous over the OP's idea of just having the same function able
to cope with files as well as strings. The only advantage that I can
see is that unpackf() might be made able to accept a pathlike, which
it will open, read from, and close. (Since a pathlike could be a
string, the single function would technically be ambiguous.) And I'd
drop that idea in the YAGNI basket.

> Or we could use a generic function. There is plenty of precedent for
> generic files in the stdlib. For example, zipfile accepts either
> a file name, or an open file object.
> def unpack(fmt, frm):
>     if hasattr(frm, "read"):
>          return _unpack_file(fmt, frm)
>     else:
>          return _unpack_bytes(fmt, frm)

FTR, I am +0.9 on this kind of proposal - basically "just make it
work" within the existing API. It's a small amount of additional
complexity to support a quite reasonable use-case.

> Drew has asked for a better hammer, and you're telling him he really
> wants a space shuttle.

But but.... a space shuttle is very effective at knocking nails into
wood... also, I just want my own space shuttle. Plz? Thx. Bye! :)


More information about the Python-ideas mailing list