[Python-Dev] A Subtle Bug in Class Initializations

[Erik Bray]

On Thu, Aug 9, 2018 at 7:21 PM Steve Dower <steve.dower at python.org> wrote:
>
> On 09Aug2018 0818, Erik Bray wrote:
> > On Mon, Aug 6, 2018 at 8:11 PM Eddie Elizondo <eelizondo at fb.com> wrote:
> >> 3) Special case the initialization of PyType_Type and PyBaseObject_Type within PyType_Ready to now make all calls to PyVarObject_HEAD_INIT use NULL. To enable this a small change within PyType_Ready is needed to initialize PyType_Type PyBaseObject:
> >
> > Coincidentally, I just wrote a long-ish blog post explaining in
> > technical details why PyVarObject_HEAD_INIT(&PyType_Type) pretty much
> > cannot work, at least for extension modules (it is not a problem in
> > the core library), on Windows (my post was focused on Cygwin but it is
> > a problem for Windows in general):
> > http://iguananaut.net/blog/programming/windows-data-import.html
> >
> > The TL;DR is that it's not possible on Windows to initialize a struct
> > with a pointer to some other data that's found in another DLL (i.e.
> > &PyType_Type), unless it happens to be a function, as a special case.
> > But &PyType_Type obviously is not, so thinks break.
>
> Great write-up! I think logically it should make sense that you cannot
> initialize a static value from a dynamically-linked library, but you've
> conclusively shown why that's the case. I'm not clear whether it's also
> the case on other OS's, but I don't see why it wouldn't be (unless they
> compile magic load-time resolution).

Thanks!  I'm not sure what you mean by "on other OS's" though.  Do you
mean other OS's that happen to use Windows-style PE/COFF binaries?
Because other than Windows I'm not sure what we care about there.

For ELF binaries, at least on Linux (and probably elsewhere) it the
runtime loader can perform more sophisticated relocations when loading
a binary into memory, including relocating pointers in the binary's
.data section.  This allows it to initialize data in one executable
"A" with pointers to data in another library "B" *before* "A" is
considered fully loaded and executable.

So this problem never arises, at least on Linux.

> > So I'm +1 for requiring passing NULL to PyVarObject_HEAD_INIT,
> > requiring PyType_Ready with an explicit base type argument, and maybe
> > (eventually) making PyVarObject_HEAD_INIT argumentless.
>
> Since PyVarObject_HEAD_INIT currently requires PyType_Ready() in
> extension modules already, then don't we just need to fix the built-in
> types?
>
> As far as the "eventually" case, I'd hope that eventually extension
> modules are all using PyType_FromSpec() :)

+1 :)