[Python-Dev] adding Construct to the standard library?

[tomer filiba]

Giovanni Bajo:
> Both ctypes and construct provide a way to describe a
> binary-packed structure in Python terms: and this is an overload of
> functionality
so does struct, so why not just use struct? there's a receipe at the python
cookbook that adds "naming ability" to fields, i.e.

">6s.destincation 6s.source H.type"

something like that, so when you parse you get named attributes instead of
a tuple. so why did ctypes wrote another mechanism? because you can't extend

it and you can't nest it. but ctypes, just as well, provides the mechanisms
for its
requirements -- defining C structs, not arbitrarily complex structures. so
of course
it doesnt, and shouldnt, support variable-length fields or data pointers,
which are
common  in file formats, protocols, and other complex data structures --
what
you can't do with a C struct you don't need to do with ctypes.

----

now i'll save me a mail and put this also here:

Greg Ewing:

> It does seem rather silly to have about 3 or 4
> different incompatible ways to do almost exactly
> the same thing (struct, ctypes, NumPy and now
> Construct).

* struct is designed for packing and unpacking, but is very limited
* ctypes is not oriented at packing/unpacking, it only provided a
mechanism to handle its requirements, which are domain specific
and not general purpose.
* i never checked how NumPy packs arrays, etc., but it's also
domain-specific, as it's a math library, not a generic packer/unpacker.

and trust me those are not the only ones. the reason people have
to *reinvent the wheel* every time is the lack of a *generic*
parsing/building
library. (i prefer the term parsing over unpacking. check my blog for more
details)

yes, putting bytes together isn't too complicated, and because people
don't have a built-in mechanism for that, they tend to just "oh, well,
it can't be too complicated, i'll just write one for me", and this yields
many
flavors of packers/unpackers, all incompatible.

Construct is the first library, that i'm aware of, that is dedicated to
parsing/building, instead of doing it as a side-kick domain-specific
mechanism.

Construct is a *superset* of all those packers and unpackers, and
had it been part of stdlib, people would have used it instead.

of course it's only been released a month ago, and couldnt have been
already included in the stdlib, i still think it has a room there. existing
projects can be ported without too much effort, and new ones could
benefit from it as well.


-tomer


On 4/19/06, Giovanni Bajo <rasky at develer.com> wrote:
>
> tomer filiba <tomerfiliba at gmail.com> wrote:
>
> > the point is -- ctypes can define C types. not the TCP/IP stack.
> > Construct can do both. it's a superset of ctype's typing mechanism.
> > but of course both have the right to *coexist* --
> > ctypes is oriented at interop with dlls, and provides the mechanisms
> > needed for that.
> > Construst is about data structures of all sorts and kinds.
> >
> > ctypes is a very helpful library as a builtin, and so is Construct.
> > the two don't compete on a spot in the stdlib.
>
>
> I don't agree. Both ctypes and construct provide a way to describe a
> binary-packed structure in Python terms: and this is an overload of
> functionality. When I first saw Construct, the thing that crossed my head
> was:
> "hey, yet another syntax to describe a binary-packed structure in Python".
> ctypes uses its description to interoperate with native libraries, while
> Construct uses its to interoperate with binary protocols. I didn't see a
> good
> reason why you shouldn't extend ctypes so to provide features that it is
> currently missing. It looks like it could be easily extended to do so.
>
> Giovanni Bajo
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060419/83e806af/attachment.html