Upgrading Python Breaks Extensions; Fix proposal

[Ignacio Vazquez-Abrams]

On 28 Aug 2001, Bryan Olson wrote:

> I've done a Google search and found that others have had the same
> problem I'm having.  Any upgrade of Python, even from 2.0 to 2.1, breaks
> extension modules.  I'm working on MS-Windows, and the extension cannot
> find python20.dll, which is not surprising because it was replace with
> python21.dll.  I suppose I could put back the old DLL, but I don't know
> what that would do, since the rest of the system is using the newer DLL.
>
> I don't the workings of Windows DLL's are to blame; Unix shared
> libraries would have the same problem.

Actually they don't. In the Unix version the .so's link back to the
executable; presumably they don't do that in Windows. Does anyone know why?

> If the interface to Python extensions actually changed, then certainly
> there's no avoiding updating the extensions.  The current system is much
> worse; any upgrade to Python breaks extensions, in many cases
> needlessly.
>
> I don't think that giving different versions of dynamic libraries the
> same name, such as "python.dll", is a good idea.  It would just cause
> other problems.

Not nearly as many problems as 5 different versions of msvcrt40.dll ;)

> So here's my proposal for a solution: don't make the extension load the
> library.  When the interpreter loads the extension, it should give it a
> pointer through which it can find all the library objects. The pointer
> could just be the system-dependent library pointer/handle, but more
> elegant would be a table of functions, defined the same on all
> platforms.

What should happen is that the extensions under Windows should have the same
behaviour as extensions on Unix; they should reach back to the executable for
run-time bindings. IIRC, the DllMain() function is passed the handle of the
executable when the DLL is loaded; at that point the bindings should be done.

Obviously this makes the DllMain() function a little more complex than it can
be, but such is life.

> I think the change could be made without forcing code changes to
> extensions (though of course they'd need one last re-compile).  We just
> have to change the statically-linked stubs library.  We need one
> function that always gets linked in, and that the interpreter will call
> with the function-table pointer.  The utility functions that the library
> offers would be stubs that get the function pointers from the table, and
> call through them.
>
> Of course we should think this through and try to find a way to
> negotiate versions, so that extensions last as long as possible.
>
>
> Is this idea PEP-worthy?
>
> --Bryan

I think that making Python behave the same under different operating systems
is _always_ PEP-worthy, but that's just me.

-- 
Ignacio Vazquez-Abrams  <ignacio at openservices.net>