[Tutor] How to make a proposal for module change to Python struct module?

[Magnus Lyckå]

At 19:55 2003-06-13 -0700, Scott Chapman wrote:
>I'd like to propose an addition to the struct module, of a 'z' format
>indicator to indicate a zero/null terminated string.

Whether you use 'p' or 's' when you pack data, Python
will fill up the remaining space with nulls. If you
create the strings in Python and unpack them in Python
and don't want a trailing string of nulls, use 'p', not
's', or do .replace('\0','') on the string. (Note that
Python doesn't consider \0 to be whitespace, so .strip()
or .rstrip() won't work.

>How are these proposals made?

For big things: Write a PEP. See the Python site. For smaller
issues, it's probably better to just ask on the main python
mailing list a.k.a. comp.lang.python, or on the python
developers mailing list if you don't feel that you want a
broader discussion first. I guess you can also file a feature
request on Sourceforge.

>(or is 's' format with no preceeding number going to accomplish this?)

It's probably a good idea to make a proper test before you
suggest a language change. ;) Actually, I think you should try
out such a thing instead of aking here. It takes about fifteen
seconds to check if you have Python running. (And you always
have, right? ;) It's not more typing than you do when you write
the question. As you can thus quickly verify, an 's' without a
preceeding number will just store the first value in the provided
string when you pack. On unpack, you will get an error unless your
format string has a size that matches the packed string.

For the Python development process, see:
http://www.python.org/dev/
http://www.python.org/peps/

If your objective is to allow structs to vary in size, I don't
think you will convince anyone. As you see if you do

import struct
help(struct)

"Functions to convert between Python values and C structs."

A C struct is fixed in size, right? It can't contain character
arrays of variable size that are malloc'd, only pointers to them.
Any "string" actually stored in the struct will have an allocated
size. This is the data structure the struct module intends to
capture. You will hardly convince anyone into making it into some
generic converter for binary data.

If your objective is just to avoid having to do .replace('\0', '')
you might possibly get agreement to that. I suppose it's more
likely that you get it included if you actually implement the
code that does it... :)

The code in question is written in C and called structmodule.c.
The relevant part for unpacking is in the end, and looks like
this:
         ...
         if (c == 's') {
                 /* num is string size, not repeat count */
                 v = PyString_FromStringAndSize(str, num);
                 if (v == NULL)
                         goto fail;
                 str += num;
                 num = 0;
         }
         ...

I guess you should follow this up with (c == 'z') and just find
the first null in 'str' before you call PyString_FromStringAndSize
so that you can find a proper value for the second paramter to
use instead of num.

For packing, s and z should do the same thing, right?

Besides this coding, tests should be written, and the library manual
and source code doc strings should be updated.

If you offer to do these things, I think it's likely that it will
be accepted even if there is no big desire from other people to have
it. But if you expect someone else to do the work, I really think
you need to convince Guido that there is a real need for this feature.
No python core developers have too little to do, and plenty of free
time... Writing code and docs is probably much better spent time than
arguing a case anyway... Both for learning, for sense of fulfillment,
for the impression you make on the Python community and above all, for
the chances of getting it into the code...


--
Magnus Lycka (It's really Lyckå), magnus@thinkware.se
Thinkware AB, Sweden, www.thinkware.se
I code Python ~ The Agile Programming Language