[issue12927] test_ctypes: segfault with suncc

[Meador Inge]

Meador Inge <meadori at gmail.com> added the comment:

I see what is going on.  I don't think the 'libffi' code can be executed correctly when built by 'suncc'.  The code currently requires '__attribute__((regparm(1)))', which AFAICT is not implemented by 'suncc';  only 'gcc'.

Take a look at 'ffi_closure_SYSV_inner' in 'Modules/_ctypes/libffi/src/x86/ffi.c':

void FFI_HIDDEN ffi_closure_raw_SYSV (ffi_raw_closure *)
     __attribute__ ((regparm(1)));

...

/* This function is jumped to by the trampoline */

unsigned int FFI_HIDDEN
ffi_closure_SYSV_inner (closure, respp, args)
     ffi_closure *closure;
     void **respp;
     void *args;
{
  /* our various things...  */
  ffi_cif       *cif;
  void         **arg_area;

  cif         = closure->cif;
  arg_area    = (void**) alloca (cif->nargs * sizeof (void*));

The '__attribute__ ((regparm(1)))' causes the first parameter to be passed in a register and the rest of the parameters to be spilled to the stack.  For most (all?) x86 compilers, '%eax' is the first register used to pass parameters when the calling convention requires registers.

'suncc' ignores this attribute (you can see warnings by greping the build logs) and expects 'closure' to be on the stack where as the trampoline code puts it in '%eax'.  This causes a bogus 'closure' and a huge value for 'cif->nargs', which overflows the stack with 'alloca'.

If you build Python on OpenSolaris with 'gcc' (pkg install gcc-dev), then everything works fine.  Does 'libffi' claim to support Solaris Studio?  If not, then this can be closed.

Also see issue1544339.  This issue was run into there as well (although no one ever really diagnosed it).

----------
nosy: +meadori

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12927>
_______________________________________