[Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned

[noreply@sourceforge.net]

Bugs item #472568, was opened at 2001-10-18 14:31
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyBuffer_New() memory not aligned

Initial Comment:
Memory buffer areas created by PyBuffer_New are
missaligned on 32 bit machines, for doubles (64 bit).

This typically generates a bus error crash on most RISC
machines when a library function tries to write a
double in a memory buffer allocated by PyBuffer_New().

When looking at the bufferobject.c code, this seems to
come from the fact that  the memory buffer points at
'malloc() + sizeof  struct PyBufferObject'; 	[line:
b->b_ptr = (void *)(b + 1); ]

To the extent that struct PyBufferObject is not a
multiple of sizeof (double), the largest value type;
thus the misalignment, and the crashes on most RISC
processors.

A quick and temporary solution would consist in adding
a dummy double field as the last field of the
PyBufferObject structure:

struct PyBufferObject {
...
double dummy; 
} ;

and setting
b->b_ptr = (void*)&b->dummy;  /*was (void*(b+1)*/

In doing so, b->b_ptr will always by aligned on sizeof
(double), on either 32 and 64 bit architectures.


Since I'm on the buffer type problem: It would be nice
(and probably easy to do) to augment the buffer
protocol interface with the ability to specify the
basic value type stored in the buffer.

A possible list of values (enum ...) would be:

- undefined (backward compatibility)
- char, unsigned char, short, ....int, ... long, long
long, float, double, and void* (memory address).

This would enable to check at runtime the type of
values stored in a buffer (and prevent missalignement
buserrors, as well as catching garbage situations when
improper array types are passed by means of the buffer
interface [e.g.: int/float/double/short...).

Frederic Giacometti


Frederic Giacometti






----------------------------------------------------------------------

>Comment By: Tim Peters (tim_one)
Date: 2001-10-31 17:31

Message:
Logged In: YES 
user_id=31435

The only portable way to fix this (assuming it's broken -- 
I don't see any alignment promises in the docs, and since 
we never use it we can't mine the source code for clues 
either) is to have PyBuffer_New do a second separate malloc
(size) and set b_ptr to that.  The C std guarantees memory 
returned by malloc is suitably aligned for all purposes; it 
doesn't promise that any standard type captures the 
strictest alignment requirement (indeed, at KSR we returned 
128-byte aligned memory from malloc, to cater to 
our "subpage" type extension).

----------------------------------------------------------------------

Comment By: Frederic Giacometti (giacometti)
Date: 2001-10-31 16:31

Message:
Logged In: YES 
user_id=93657

A portable solution (im[provement over what I proposed)
would consitst in declaring 'dummy' with a union type,
'unionizing' all C-ANSI value types (and including 'long
long' optionally by mean of an #ifdef).

 { .... union { int Int; long Long; double Double; void*
Pvoid ...} dummy; }

All (void*)obj->dummy can be replaced with obj->dummy.Pvoid

FG


----------------------------------------------------------------------

Comment By: Jeremy Hylton (jhylton)
Date: 2001-10-22 13:19

Message:
Logged In: YES 
user_id=31392

Note to whomever take this bug: PyBuffer_New() is not called
anywhere in the Python source tree; nor are there any tests
for buffer objects that I'm aware of.  A few simple test
cases would have caught this bug already.  (And for the case
of the builtin buffer() call, it might be good if it used
PyBuffer_New().)


----------------------------------------------------------------------

Comment By: Frederic Giacometti (giacometti)
Date: 2001-10-18 14:35

Message:
Logged In: YES 
user_id=93657

I wasn't looged in when I submitted the item. Don't think
I'm becoming anonymous :))


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470