[Python-Dev] --with-fpectl changes the CPython ABI

[Nathaniel Smith]

Hi all,

Well, we finally got that ucs2/ucs4 stuff all sorted out (yay), but I
just learned that there's another CPython build flag that also changes
the ABI: --with-fpectl

Specifically, it seems that if you build CPython with --with-fpectl,
and then use that CPython to build an extension module, and that
extension module uses PyFPE_{START,END}_PROTECT (like e.g. Cython
modules do), and you then try to import that extension module on a
CPython that *wasn't* built with --with-fpectl, then it will crash.
This bug report has more gory details:
    https://github.com/numpy/numpy/issues/8415

The reverse is OK -- extensions built in a no-fpectl CPython can be
imported by both no-fpectl and yes-fpectl CPythons.

So one consequence is easy -- we need to make a note in the manylinux1
definition saying that you have to use a no-fpectl CPython build to
make manylinux1 wheels, because that's the only way to guarantee
compatibility. I just submitted a PR for this:
  https://github.com/python/peps/pull/166
(Fortunately the manylinux1 docker images are already set up this way,
so in practice I think everyone is already doing this.)

But... in general this is kind of an unfortunate issue, and it's not
restricted to Linux. Should we do something? Some options:

Add another ABI flag -- e.g. cp35dmf vs. cp35dm? Though AFAICT the
offending macros are actually part of the allegedly stable ABI (!!),
so I guess this isn't really a solution. (I'm not 100% confident about
how to tell whether something is part of the stable ABI, but: Python.h
unconditionally imports pyfpe.h, and pyfpe.h doesn't have any
Py_LIMITED_API checks.)

Drop support for fpectl entirely in 3.7 on the grounds that it's not
worth the trouble? (The docs have said "don't use this" at the top
forever[1], and after clicking through every hit on github code search
for language = Python and string "turnon_sigfpe" [2], I found exactly
4 non-documentation usages [3], all of which are already broken in
no-fpectl builds.) We obviously can't do this in a point release
though, because there are lots of extant extension modules referencing
these symbols.

Or maybe make it so that even no-fpectl builds still export the
necessary symbols so that yes-fpectl extensions don't crash on import?
(This has the advantage that it can be done in a point release...)

Thoughts?

-n

[1] https://docs.python.org/2/library/fpectl.html
[2] https://github.com/search?l=Python&p=1&q=turnon_sigfpe&type=Code&utf8=%E2%9C%93
[3]
  https://github.com/podhrmic/JSBSim/blob/36de9ac63c959cef5d7b2c56fb49c1a57fd46369/tests/CheckScripts.py#L28
  https://github.com/tmbdev/iuprlab/blob/239918b5ec0f8deecbc7c2ec1283a837d11a7b5a/boostedmlp.py#L161
  https://github.com/wcs211/BEM-3D-Python/blob/874aaeffc3dac5f698f875478c3accf2b5a14daf/FSI_bem3d.py#L25
  https://github.com/neobonzi/SoundPlagiarism/blob/7cff7f0145217bffb3a3cebd59a946feee23aff6/processor.py#L31

-- 
Nathaniel J. Smith -- https://vorpus.org