Tim Peters <tim_one at email.msn.com> wrote in message
news:000301bf1545$76b71560$ed2d153f at tim...
> [Tim, asks why the xstruct extension
> http://www.sis.nl/python/xstruct/xstruct.html
> is written in C
> ]
>
> [Robin Boerdijk]
> > There are a number of reasons for implementing the xstruct module in C
as
> > opposed to implementing it as a Python wrapper around the current struct
> > module.
> >
> > ... [code to randomly change the value of a field somewhere in the
middle
> > of a binary string] ...
>
> But that's not a reasonable implementation -- the data only needs to be in
> packed form when it's made externally visible; e.g., Just van Rossum's
> sstruct module got toward a similar end by slinging ordinary class
instance
> dicts:
>
> http://starship.python.net/crew/just/code/sstruct.py
>
> Mutations are faster that way, despite being written in Python. Just pays
> for a regular struct.pack() at the end (and/or struct.unpack() at the
> start), though.
I have done a comparison on how fast sstruct is compared to xstruct using
the Xsdp example that comes with xstruct. The test scripts are at the end of
this post. They both do 10000 iterations of creating and packing an
XsdpMessage. On my computer, the sstruct script takes 2.4 seconds, the
xstruct script takes 0.33 seconds.
> It would help if you beefed up the docs. For example, none of the
module's
> magic constants are documented (whether for layout, field type or
"flags");
> doc strings are generally absent; and objects of type structdef and
> structobject respond to dir() by returning an empty list. Because of
this,
> you really have to be an expert in the use of the current struct module to
> approach the xstruct module, and there's no way to figure it out except by
> reading the implementation code.
You are right. So far, I have only documented up to the point that I thought
that people who need to interact intimately with C structs from Python (and
have been struggling with struct.pack and struct.unpack to do that) would
get enthousiastic about this new module.
> You could not at present, although it's not clear that's a bad thing. The
> buffer interface has had a troubled history, and it's unclear whether it
> will survive.
That's a problem. My next step would have been to add buffer support to
cStringIO and socket to make it all work together. Just imagine the amount
of string copies this would save.
> It's not ready for prime time (e.g., it's easy to crash the interpreter
with
> the little that's already there, staying within pure Python).
Can you give me an example ?
> mucking-with-raw-c-structs-is-a-thing-to-avoid-ly y'rs - tim
And that's exactly why I want this to be implemented properly as a Python
extension module !!!!
Robin
-- test script for sstruct ---
import sstruct
import time
import copy
XsdpMessageFormat = """
>
magic:4s
major_version:B
minor_version:B
byte_order:B
message_type:B
correl_id:L
data:16s
"""
default_msg = {}
default_msg['magic'] = "XSDP"
default_msg['major_version'] = 1
default_msg['minor_version'] = 0
default_msg['byte_order'] = 0
before = time.clock()
for i in xrange(0, 10000):
msg = copy.copy(default_msg)
msg['correl_id'] = 0x01020304
msg['message_type'] = 1
msg['data'] = "Hello, World !"
sstruct.pack(XsdpMessageFormat, msg)
after = time.clock()
print "10000 iterations in", after - before, "seconds"
--- test script for xstruct ---
import xstruct
import time
XsdpMessage = xstruct.structdef(xstruct.big_endian, [
("magic", (xstruct.string, 4), "XSDP", xstruct.readonly),
("version", (xstruct.octet, 2), (1, 0)),
("byte_order", (xstruct.octet, 1), 0, xstruct.readonly),
("message_type", (xstruct.octet, 1)),
("correl_id", (xstruct.unsigned_long, 1)),
("data", (xstruct.string, 16))
])
before = time.clock()
for i in xrange(0, 10000):
msg = XsdpMessage()
msg.correl_id = 0x01020304
msg.message_type = 1
msg.data = "Hello, World !"
after = time.clock()
print "10000 iterations in", after - before, "seconds"