[New-bugs-announce] [issue29328] struct module should support variable-length strings

[Elizabeth Myers]

New submission from Elizabeth Myers:

There was some discussion on python-ideas about this, and I figured it would be more productive to bring it here since to me this appears to be a glaring omission.

The struct module has no capability to support variable-length strings; this includes null-terminated and Pascal-ish strings with a different integer datatype (usually in binary) specifying length.

This unfortunate omission makes the struct module extremely unwieldy to use in situations where you need to unpack a lot of variable-length strings, especially iteratively; see https://mail.python.org/pipermail/python-ideas/2017-January/044328.html for why. For zero-terminated strings, it is essentially impossible.

It's worth noting many modern protocols use variable-length strings, including DHCP.

I therefore propose the following extensions to the struct module (details can be bikeshedded over :P):

- Z (uppercase) format specifier (I did not invent this idea, see https://github.com/stendec/netstruct - although that uses $), which states the preceding whole-number datatype is the length of a string that follows.
- z (lowercase) format specifier, which specifies a null-terminated (also known as C style) string. An optional length parameter can be added to specify the maximum search length.

These two additions will make the struct module much more usable in a wider variety of contexts.

----------
components: Library (Lib)
messages: 285828
nosy: Elizacat
priority: normal
severity: normal
status: open
title: struct module should support variable-length strings
type: enhancement
versions: Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29328>
_______________________________________