[Python-3000] PyObject_HEAD_INIT

[Adam Olsen]

On Fri, Nov 21, 2008 at 10:53 AM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 2008-11-21 18:36, Adam Olsen wrote:
>> On Fri, Nov 21, 2008 at 7:14 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>>> On 2008-11-20 20:34, Roger Binns wrote:
>>>> M.-A. Lemburg wrote:
>>>>> Whether you write:
>>>>> {PyObject_HEAD_INIT(0), 0, ...
>>>>> or
>>>>> {PyVarObject_HEAD_INIT(0, 0), ...
>>>>> for your type definition doesn't really make much difference.
>>>> Actually in Py 3 it does.  If you use the former (which is how Py 2 does
>>>> it) then you get serious compiler warnings due to misaligned fields in
>>>> Py 3 and presumably even worse if run the code.
>>> You might get warnings (esp. from GCC), but I have yet to see a compiler
>>> that doesn't map the above to the same memory.
>>>
>>> After all, Python 2 has been using this layout for years without any
>>> compiler warnings or segfaults because of this.
>>
>> The definition of PyObject_HEAD_INIT and PyVarObject_HEAD_INIT
>> changed.  We've gone from a series of common fields to a single struct
>> containing the fields.  With the series of fields using
>> PyObject_HEAD_INIT followed by a size was perfectly correct, but with
>> a struct it's gibberish.
>
> Sigh...
>
> Python 2.5:
> -----------
>
> (gdb) print PyUnicode_Type
> $2 = {ob_refcnt = 1, ob_type = 0x6404c0, ob_size = 0, tp_name = 0x505972 "unicode",
>  tp_basicsize = 48, tp_itemsize = 0, tp_dealloc = 0x470500 <unicode_dealloc>,
> tp_print = 0,
>  tp_getattr = 0, tp_setattr = 0, tp_compare = 0, tp_repr = 0x4746c0 <unicode_repr>,
>  tp_as_number = 0x644f60, tp_as_sequence = 0x644f00, tp_as_mapping = 0x644ee0,
> ...
>
> (gdb) print &PyUnicode_Type.tp_name
> $3 = (const char **) 0x642758
> (gdb) print &PyUnicode_Type.ob_refcnt
> $4 = (Py_ssize_t *) 0x642740
>
> Python 3.0:
> -----------
>
> (gdb) print PyUnicode_Type
> $1 = {ob_base = {ob_base = {ob_refcnt = 1, ob_type = 0x733940}, ob_size = 0},
>  tp_name = 0x4f52e9 "str", tp_basicsize = 56, tp_itemsize = 0,
>  tp_dealloc = 0x42af80 <unicode_dealloc>, tp_print = 0, tp_getattr = 0,
> tp_setattr = 0,
>  tp_compare = 0, tp_repr = 0x431ca0 <unicode_repr>, tp_as_number = 0x736420,
> ...
>
> (gdb) print &PyUnicode_Type.tp_name
> $3 = (const char **) 0x735bf8
> (gdb) print &PyUnicode_Type.ob_base
> $4 = (PyVarObject *) 0x735be0
>
> In both cases, the fields are 24 bytes apart (on my 64-bit machine).
>
> Yes, it's a different way of writing and accessing the resp. fields.
> No, it's not a different memory layout.
> Yes, this is binary compatible.
> No, this is not going to help you, since the rest of Python 3 is not ;-)

You're comparing already compiled code.  The issue is with
recompiling, ie source compatibility.

In 2.5 the macros expanded to look like this:

PyTypeObject PyUnicode_Type = {
    1,                              /* ob_refcnt */
    &PyType_Type,              /* ob_type */
    0,                                    /* ob_size */
    "unicode",                       /* tp_name */
    sizeof(PyUnicodeObject),      /* tp_basicsize */

Try the same macro in 3.0 and it'll look like this:

PyTypeObject PyUnicode_Type = {
    {
        {
            1,                              /* ob_refcnt */
            &PyType_Type,              /* ob_type */
        },
        /* Trailing ob_size gets implicitly initialized to 0 */
    },
    0,                                        /* ob_size?  Nope, tp_name! */
    "unicode",                      /* tp_name?  Nope, tp_basicsize! */
    sizeof(PyUnicodeObject),            /* tp_basicsize?  Nope, tp_itemsize! */

The compiler knows what layout a PyTypeObject should have, but the
initializer doesn't match up.

-- 
Adam Olsen, aka Rhamphoryncus