[Python-ideas] Ideas for improving the struct module

[Elizabeth Myers]

On 20/01/17 10:59, Paul Moore wrote:
> On 20 January 2017 at 16:51, Elizabeth Myers <elizabeth at interlinked.me> wrote:
>> Should I write up a PEP about this? I am not sure if it's justified or
>> not. It's 3 changes (calcsize and two format specifiers), but it might
>> be useful to codify it.
> 
> It feels a bit minor to need a PEP, but having said that did you pick
> up on the comment about needing to return the number of bytes
> consumed?
> 
> str = struct.unpack('z', b'test\0xxx')
> 
> How do we know where the unpack got to, so that we can continue
> parsing from there? It seems a bit wasteful to have to scan the string
> twice to use calcsize for this...
> 
> A PEP (or at least, a PEP-style design document) might capture the
> answer to questions like this. OTOH, the tracker discussion could
> easily be enough - can you put a reference to the bug report here?
> 
> Paul
> 

Two things:

1) struct.unpack and struct.unpack_from should remain
backwards-compatible. I don't want to return extra values from it like
(length unpacked, (data...)) for that reason. If the calcsize solution
feels a bit weird (it isn't much less efficient, because strings store
their length with them, so it's constant-time), there could also be new
functions that *do* return the length if you need it. To me though, this
feels like a use case for struct.iter_unpack.

2) I want to avoid making a weird incongruity, where only
variable-length strings return the length actually parsed. This also
doesn't really help with length calculations unless you're doing
calcsize without the variable-length specifiers, then adding it on. It's
just more of an annoyance.


On 20/01/17 12:18, Guido van Rossum wrote:
> I'd be wary of making a grab-bag of small improvements, it encourages
> bikeshedding.
>
> --Guido (mobile)

Definitely would prefer to avoid a bikeshed here, though other
improvements to the struct module are certainly welcome!

(Though about a better interface, I made a neat little prototype module
for an object-oriented interface to struct, but I want to clean it up
before I release it to the world... but I'm not sure I want to include
it in the standard library, that's for another day and another proposal :p).

--
Elizabeth