[ python-Bugs-1433886 ] pointer aliasing causes core dump, with workaround

[SourceForge.net]

Bugs item #1433886, was opened at 2006-02-17 16:56
Message generated for change (Comment added) made by qbarnes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1433886&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Quentin Barnes (qbarnes)
Assigned to: Tim Peters (tim_one)
Summary: pointer aliasing causes core dump, with workaround

Initial Comment:
When building 2.3.3 on Solaris using Studio 11 and
"-xalias_level=std" to enable ISO C99 pointer aliasing
rules, the interpreter would core dump when running
test_descr2.py.  From examining the source, I believe
this problem exists in 2.4.2 as well, but did not
verify it.

I was able to simplify the code to invoke the problem
down to:
====
class C1(object):
   def __new__(cls):
       return super(C1, cls).__new__(cls)

a = C1()
====

I tracked the problem to super_init() in
Objects/typeobject.c.

The local variable "obj" is of type "PyObject *" and
"type" is of type "PyTypeObject *".  In this function,
both variables can be pointing to the same object in
memory.  Since the pointers are not of compatible
types, the compiler ends up optimizing out a memory
load between Py_INCREF(obj) and Py_INCREF(type) caching
a previously read value.  This causes the object
reference counter to only be incremented once instead
of twice.  When the object is deallocated, the
interpreter blows up.

A workaround is to cast one of the pointers to the
others type so that the compiler can see the pointer
aliasing potential.  This is what I did in the patch.

What I suspect needs to be done as a more global fix is
to discontinue use of the PyObject_VAR_HEAD macro hack.
 Casting between structures containing this macro
creates ISO C99 pointer aliasing violations.  The
contents of the macro needs to be in its own structure
which is referenced so a compliant compiler can know of
 possible aliasing.

----------------------------------------------------------------------

>Comment By: Quentin Barnes (qbarnes)
Date: 2006-08-03 21:24

Message:
Logged In: YES 
user_id=288734

There is no reason why the source code for Python cannot
conform to ISO C99 pointer aliasing rules.  If pointers are
being improperly aliased violating the C standard, it's just
bad programming practice.

The only reason to use a switch like GCC's
-fno-strict-aliasing or Solaris' -xalias_level=any is for
old legacy code, not well-programmed, actively maintained code.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-08-03 16:24

Message:
Logged In: YES 
user_id=11375

I have the vague impression that Python's code doesn't
conform to strict aliasing, and it therefore must always be
compiled with GCC's -fno-strict-aliasing option or the
equivalent.  Your fix might avoid that one problem, but
there might be other subtle bugs. 

Assigning to Tim Peters, who will know if the
-fno-strict-aliasing statement is correct.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1433886&group_id=5470