[Python-Dev] Struct module format code addition (previously was in the subprocess thread)

[Josiah Carlson]

> > People would be spending all their time trying to gather groups of
> > people to agree with them that functionality K is necessary.
> 
> And that would be a good thing, atleast in the specific case. Even
> if it is undebated that the feature (convert numbers into odd-sized
> byte strings) is desirable, the specific API needs careful
> consideration, since it cannot be changed easily after it has been
> added. So all these new features are a serious legacy if they later
> turn out to be ill-designed.

If it turns out that the feature is ill-designed, then there is a
standard method of dealing with it:
1. Declare that it will be Deprecated in the future.
2. Start raising a DeprecationWarning for at least one major version of
Python.
3. Remove the functionality 

In the case of struct and this addition, if it is the case that this
modification is catastrophic to Python (very unlikely), I would hope
that someone would have said, "hey, this breaks applications A, B, C, D, ...,
and it inclusion was the biggest mistake Python has ever made, even
moreso than anyone ever responding to Josiah* in the first place."

* Me trying to be funny.

> > I'm not even saying that we should add a new module.  I'm not even
> > saying we should add a new function to a module.  Heck, I'm not even
> > asking for a new argument to a function that previously existed.  I am,
> > quite literally, asking for the equivalent of less than one bit in a
> > flag (there are currently 22 format/endian characters, which are all
> > orthogonal, which would require 5 bits, if they were flags).
> 
> Correct. Yet I would feel more comfortable if you had proposed a new
> module, or a new function to an existing module. The former would allow
> distribution independent of Python, and the latter would not put a
> burden on users of the existing functions.

For those who use struct as it is now, the proposed additional format
codes raise a struct.error exception in older versions of Python.  If
people are relying on pack/unpack with characters 'g' or 'G' raising a
struct.error exception, then something is definitely wrong with the way
they have been using struct.

If people want to use the modifications with a previous version of
Python, the changes are easily backported (I think one can even take
structmodule.c from CVS and compile it with an older source tree).

> Yes. The extension you propose probably does not cause backward
> compatibility problems. Strictly speaking, there are programs that
> break under this extension, but this is the case for any extension,
> and such programs likely don't occur in real life.

Agreed.


> No. For such a change, we need to study whether there alternative APIs
> which achieve the same effect (which there are), and whether the new
> flag puts an additional learning burden on users of the existing API
> (which it does).

Tim Peters' alternative long2bytes and bytes2long looked quite usable,
and indeed produces strikingly similar results, for single item packing
and unpacking scenarios.

The learning curve of struct was claimed to be steep in the case of
native byte alignments, or for those not familliar with C types or
structs.

Since these codes are not native to any platform, it doesn't make the
native byte alignment stuff any more difficult.

If people are having trouble understanding the concept of C structs, I
don't believe that two new format characters are going to be
significantly more difficult to swallow than the 17 others that already
exist.  Especially when the standard C type codes 'cbhilqspBHILQ' are
likely sufficient for their needs.  In the case of them not being
sufficient, well hey, they would have another two options that may do it
for them.


> > If every request to interpret a new flag required significant community
> > involvement, goodness, would it take an act of Guido to get a commit
> > done?
> 
> No, I have committed new features myself in the past. See the CVS log
> for details. It is this specific change I'm opposed to, at this point
> in time.

I was trying to be funny, to perhaps lighten the mood.  Hence "act of
Guido" rather than "act of God" (where the latter is a cliche).

 - Josiah