[Python-ideas] Ideas for improving the struct module

[Nathaniel Smith]

On Jan 20, 2017 09:00, "Paul Moore" <p.f.moore at gmail.com> wrote:

On 20 January 2017 at 16:51, Elizabeth Myers <elizabeth at interlinked.me>
wrote:
> Should I write up a PEP about this? I am not sure if it's justified or
> not. It's 3 changes (calcsize and two format specifiers), but it might
> be useful to codify it.

It feels a bit minor to need a PEP, but having said that did you pick
up on the comment about needing to return the number of bytes
consumed?

str = struct.unpack('z', b'test\0xxx')

How do we know where the unpack got to, so that we can continue
parsing from there? It seems a bit wasteful to have to scan the string
twice to use calcsize for this...


unpack() is OK, because it already has the rule that it raises an error if
it doesn't exactly consume the buffer. But I agree that if we do this then
we'd really want versions of unpack_from and pack_into that return the new
offset. (Further arguments that calcsize is insufficient: it doesn't work
for potential other variable length items, e.g. if we added uleb128
support; it quickly becomes awkward if you have multiple strings; in
practice I think everyone who needs this would just end up writing a
wrapper that calls calcsize and returns the new offset anyway, so should
just provide that up front.)

For pack_into this is also easy, since currently it always returns None, so
if it started returning an integer no one would notice (and it'd be kinda
handy in its own right, honestly).

unpack_from is the tricky one, because it already has a return value and
this isn't it. Ideally it would have worked this way from the beginning,
but too late for that now... I guess the obvious solution would be to come
up with a new function that's otherwise identical to unpack_from but
returns a (values, offset) tuple. What to call this, though, I don't know
:-). unpack_at? unpack_next? (Hinting that this is the natural primitive
you'd use to implement unpack_iter.)

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170120/6c4517bd/attachment.html>