[ python-Bugs-1433886 ] pointer aliasing causes core dump, with workaround

[SourceForge.net]

Bugs item #1433886, was opened at 2006-02-17 16:56
Message generated for change (Comment added) made by qbarnes
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1433886&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Quentin Barnes (qbarnes)
Assigned to: Nobody/Anonymous (nobody)
Summary: pointer aliasing causes core dump, with workaround

Initial Comment:
When building 2.3.3 on Solaris using Studio 11 and
"-xalias_level=std" to enable ISO C99 pointer aliasing
rules, the interpreter would core dump when running
test_descr2.py.  From examining the source, I believe
this problem exists in 2.4.2 as well, but did not
verify it.

I was able to simplify the code to invoke the problem
down to:
====
class C1(object):
   def __new__(cls):
       return super(C1, cls).__new__(cls)

a = C1()
====

I tracked the problem to super_init() in
Objects/typeobject.c.

The local variable "obj" is of type "PyObject *" and
"type" is of type "PyTypeObject *".  In this function,
both variables can be pointing to the same object in
memory.  Since the pointers are not of compatible
types, the compiler ends up optimizing out a memory
load between Py_INCREF(obj) and Py_INCREF(type) caching
a previously read value.  This causes the object
reference counter to only be incremented once instead
of twice.  When the object is deallocated, the
interpreter blows up.

A workaround is to cast one of the pointers to the
others type so that the compiler can see the pointer
aliasing potential.  This is what I did in the patch.

What I suspect needs to be done as a more global fix is
to discontinue use of the PyObject_VAR_HEAD macro hack.
 Casting between structures containing this macro
creates ISO C99 pointer aliasing violations.  The
contents of the macro needs to be in its own structure
which is referenced so a compliant compiler can know of
 possible aliasing.

----------------------------------------------------------------------

>Comment By: Quentin Barnes (qbarnes)
Date: 2006-08-04 15:04

Message:
Logged In: YES 
user_id=288734

It doesn't matter which standard is used, C99 or C89, the
aliasing violation is the same.

An object is being accessed through two different aggregate
type aliases.  The language in C99 section 6.5 is the same
in this regards as C89 section 3.3.

(Actually, C99 is slightly looser than C89 since it allows
effective type aliasing which C89 does not have.)

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2006-08-03 21:47

Message:
Logged In: YES 
user_id=31435

Yes, Python must be compiled with -fno-strict-aliasing (or
moral equivalent).

No, that's not going to change before Python 3000 (if even
then).  All Python object structs conceptually extend the
PyObject struct, including (but not limited to) object
structs conceptually extending the PyVarObject struct (which
in turn conceptually extends the PyObject struct).  This is
uniquitous in the core source code, and in all 3rd-party
extension modules defining new Python objects in C.

Note that Python doesn't much care about C99.  It was
written against K&R C, and went nearly a decade before
requiring C89 (it took that long for C89-conforming
compilers to show up on all interesting Python platforms). 
Best guess is that it will never require C99 (because best
guess is that C99-conforming compilers will never show up on
all interesting Python platforms).

That said, I don't see anything to object to in adding
fiddly casting patches to make sure compilers don't try to
exploit the trivial optimization opportunities that assuming
no aliasing in the Python source may allow.

----------------------------------------------------------------------

Comment By: Quentin Barnes (qbarnes)
Date: 2006-08-03 21:24

Message:
Logged In: YES 
user_id=288734

There is no reason why the source code for Python cannot
conform to ISO C99 pointer aliasing rules.  If pointers are
being improperly aliased violating the C standard, it's just
bad programming practice.

The only reason to use a switch like GCC's
-fno-strict-aliasing or Solaris' -xalias_level=any is for
old legacy code, not well-programmed, actively maintained code.

----------------------------------------------------------------------

Comment By: A.M. Kuchling (akuchling)
Date: 2006-08-03 16:24

Message:
Logged In: YES 
user_id=11375

I have the vague impression that Python's code doesn't
conform to strict aliasing, and it therefore must always be
compiled with GCC's -fno-strict-aliasing option or the
equivalent.  Your fix might avoid that one problem, but
there might be other subtle bugs. 

Assigning to Tim Peters, who will know if the
-fno-strict-aliasing statement is correct.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1433886&group_id=5470