Python update trouble (2.3 to 2.4): x<<y
gmc at serveisw3.net
Sun May 21 21:22:32 CEST 2006
Thank you for all the suggestions! :-)
The C routine is almost -changing data type long for word- a copy of the
function given by a hardware manufacturer, the same code used in their
firmware to calc the checksum of every piece of data sent or received,
and that data is somewhat special: it does not contain only plain ascii
character data (only 0x01 - 0x09 as delimiters) and every data set ends
with a 0x00 null byte so its fine to calculating checksum until "\x00"
is reached. And I must get the same crc calculations. That C routine
(and a very similar PHP one too) are tested for months and work smoothly.
And of course I should use integer not long!!... we aren't anymore in
the 8 bit computer era... ;-) original code from hardware manufacturer
use words by the way. And now really don't know why I used long. I
thought I did that way 'cause see pyrex C generated glue code always
using PyInt_FromLong so thought would use the same type.
I changed the C function to use integer, and it is performing a little
bit slowly than using longs (best with long: 0.614us, best with int:
0.629us), that is maybe as I said due pyrex glueing always the return
values with PyObject* PyInt_FromLong for all kind of short int to long
data types? Anyway both results are fine, and the timming gap is
insignificant (int: 1580403 loops -> 0.973 secs, long: 1601902 loops ->
1.01 secs) as i usually never have to call more than 100,000 times when
loading a full file data backup.
As a note for the checksum algorithm used here, I thought the hardware
manufacturer implemented this some years ago for serial data
transmission, but later attached some ethernet ports and they wanted to
have the protocol compatible. Anyway as I see in the link you reply, it
seems to be based in CCITT CRC-16, but they adapted to their own
requirements -performace issues for the hardware unit?- (never 0x00, etc.)
John Machin escribió:
>I missed the point to add the *and* to workaround the long result
>I think I understand it now.
>I am timing the code once translated, so here are the results for the
>crc calculation function.
>Yes, and both of us were missing the point that the "16" means 16 bits
>wide!! It is carrying an extra 16 redundant bits that don't contribute
>to the checking ability.
>If your software *needs* to generate/check the exact same 32-bit-wide
>CRC as calculated by that C routine, then what I gave you should be OK.
> Do you have any more test values? In particular test an empty string
>and say "\x00" * 8 -- sometimes a CRC specification will do something
>special with short strings e.g. pad them put with null bytes.
>That C routine is actually, on further reading, mind-boggling!! It
>stops on a null byte, rather than working on a given length. A CRC is
>normally expected to work on any data (including binary data) whether
>it contains \x00 or not. What are you using it for??
>If you don't have to interface with other software, AND a 16-bit check
>is adequate, then at the end of this message is a 16-bit version which
>will use only ints and therefore may run faster.
>However you may wish to use a CRC32 which is supplied with Python (in
>the binascii module).
>A further note: what we have so far seems not to be the CCITT aka X.25
>standard CRC-16 , but a reflection. See
>and also if you have the time and the determination, the article by
>Ross Williams that it refers to.
>=== 16-bit version ===
>gCRC16Table = 
> global gCRC16Table
> for i in xrange(0,256):
> crc = i << 8
> for j in xrange(0,8):
> if (crc & 0x8000) != 0:
> tmp = 0x1021
> tmp = 0
> crc = (crc << 1) ^ tmp
> crc &= 0xFFFF
> crc = 0xFFFF
> for c in astr:
> crc = gCRC16Table[((crc >> 8) & 255)] ^ ((crc & 0xFF) << 8) ^
> assert 0 <= crc <= 0xffff # remove when doing timings :-)
> return crc
>test = "123456asdfg12345123"
>result = CalcCRC16(test)
>print result, hex(result)
More information about the Python-list