version-number dependency in extensions (was Re: ConfigParser module in 1.52 is buggy?)

[Tim Peters]

[posted & mailed]

[Alex Martelli, about life on Windows]
> ...
> Programs that embed Python, and Python extensions, there depend on
> "python15.dll" and need to be recompiled and relinked even if nothing
> but the DLL's name has changed.  As has been wistfully noted other
> times, it WOULD be nice to remove that DLL-name dependency:
> particularly with the current rate of twice-yearly releases, Python
> users on Windows who don't recompile the extensions they use, but
> rather depend on others releasing pre-built binaries, have a hard
> time keeping up-to-date.

Mark Hammond's signoff is essential for any Windows change here, since it's
his scheme, he spent a lot of time refining it over the years, but no time
documenting it <0.5 wink>.

Someone shipping a serious Python app on Windows is going to bundle the
Python DLL it was tested with, and this scheme has at least been effective
in ensuring they don't pick up a different one by mistake.  Maybe that
shouldn't be needed "in theory", but in reality I don't know of any
cross-release Python binary compatibility testing done by anyone on any
platform.  Of course PYTHON_API_VERSION gets bumped when we *know* it's
going to be incompatible, but, e.g., I wouldn't bet a business on that no
incompatibility was overlooked between 18-Jan-1997 and 14-Mar-2000 (the
longest historical gap between PYTHON_API_VERSION increments).

> One conceptual scheme to enable that was suggested (a year ago, as I
> recall) by imitation of what TCL does (on Windows, at least; I used
> to be knowledgeable about Unix so's, too, but it has been years -- I'm
> not current any more and may have forgotten key details).  A single
> python. dll, (or maybe one per _major_ release to allow backwards
> incompatibilities -- say python2. dll, python3.dll, &c) exposes a
> single entry point, say 'pythmen' ("the fundament", in Greek:-).  All
> functions currently exposed by the Python-C API become (in Python.h
> &c, and on relevant platforms only) macros expanding to
> (*(pythmen(23)))(a,b,c) i.e. , pythmen(n) returns the function
> pointer for the n-th entry point. That wouldn't work as is with ISO C
> prototypes, so a further cast would no doubt be needed to make what
> pythmen returns into the right type of pointer to function, but that's
> a SMOP:-)

I must be missing the point.  If the DLL name doesn't change anymore, and
subsequent releases are "compatible enough" that the collection of C API
function names and signatures doesn't change either, what does this
indirection buy (as opposed to letting the linker resolve the function names
itself)?

> As presented, this would be substantial overhead, but it may be
> possible to pare it down by making pythmen into a function to be
> called once only to prepare (or at least expose) an array of function
> pointers -- if this only needs to target Windows (i.e., Unix, Mac &c
> are already free of this dependence on DLL name -- I'm not sure, are
> they?),

Don't know anything about Macs; the Unix releases do not change the names; I
expect Mark is more concerned about complex Windows-specific COM scenarios,
which I'll leave to him to explain (my brain always explodes before I fully
grasp all the things he's guarding against!).

> the data might be directly exposed by the DLL, so the macros would
> do (*(pythmen[23]))(a,b,c), again with the appropriate casting -
> - the current Tcl solution to this should be studied in detail (the Tcl
> language has its issues, but some of the tools &c are well worth
> imitation, IMHO -- Tk's not the ONLY usable thing about it:-).
>
> What do you think -- worth a PEP? If it's not to be a Windows-only
> thingie, I'd need consulting from experts of other platforms to
> prepare said PEP...

A PEP is *always* a fine idea <wink>.  I'd like to see Mark write a
rationale for the current Windows design too, beyond that it's convenient
for Python developers to have 20 versions of Python sitting on one box; the
pain for end users of hunting down a new .pyd for every extension they use
every time Python gets released has become a problem.  IOW, before a PEP
goes off suggesting solutions, I'd like a clearer statement of "the
problem(s)" it intends to solve.