Require a C compiler supporting C99 to build Python 3.11
Hi, I made a change to require C99 <math.h> "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11. Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7: https://www.python.org/dev/peps/pep-0007/ I modified Python 3.11 to require more C99 features of the <math.h> header: * bpo-45440: copysign(), hypot(), isfinite(), isinf(), isnan(), round() * bpo-46640: NAN After my NAN change (bpo-46640), Petr Viktorin asked me to update the PEP 7. I proposed a change to simply say that "Python 3.11 and newer versions use C99": https://github.com/python/peps/pull/2309 I would prefer to not have to give an exhaustive list of C99 features used by CPython, since it's unclear to me what belongs to C99 or to ISO C89. As I wrote before, Python already uses C99 features since Python 3.6. On my PEP PR, Guido van Rossum asked me to escalate the discussion to python-dev, so here I am :-) In "C99", the number "99" refers to the year 1999, the standard is now 23 years old: https://en.wikipedia.org/wiki/C99 In 2022, C99 is now well supported by C compilers supported by Python: GCC, clang, MSVC. I don't know if AIX XLC supports C99. AIX provides a "c99" compiler compatible with C99. It also seems like GCC is usable on AIX. I don't know if ICC supports C99. Python doesn't officially the ICC compiler, the ICC buildbots are gone a few years ago. But sometimes I make some changes to enhance the ICC support, when the change is small enough. Note: Python also uses C11 <stdatomic.h>, but it's not required: there are fallbacks for compilers which don't support it. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Mon, Feb 7, 2022 at 5:11 PM Victor Stinner <vstinner@python.org> wrote:
I made a change to require C99 <math.h> "NAN" constant [...]
There's a separate discussion topic lurking here. It's equally in need of discussion here (IMO), but it's orthogonal to the "should we require C99" discussion. I've changed the subject line accordingly to try to avoid derailing that discussion. Unlike the other things Victor mentions ("copysign", "round", etc.), the NAN macro is not required to be present by C99. Instead, the standard says that "NAN is defined if and only if the implementation supports quiet NaNs for the float type" (C99 §7.12p5). Victor is proposing in GH-31160 <https://github.com/python/cpython/pull/31160> to require the presence of the NAN macro in order for CPython to build, which under C99 is equivalent to requiring that the C float type supports quiet NaNs. That's not the same as requiring IEEE 754 floating-point, but it's not far off - there aren't many non-IEEE 754 floating-point formats that support NaNs. (Historically, there are essentially none, but it seems quite likely that there will be at least some non-IEEE 754 formats in the future that support NaNs; Google's bfloat16 format is one example.) So there (at least) three questions here: - Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation? - Should we require IEEE 754 floating-point for Python-the-language? For the first two, I'd much prefer either to not require NaNs, or to go the whole way and require IEEE 754 for CPython. Requiring NaNs but not IEEE 754 feels like an awkward halfway house: in practice, it would be just as restrictive as requiring IEEE 754, but without the benefits of making that requirement explicit (e.g., being able to get rid of non-IEEE 754 paths in existing code, and being able to tell users that they can reasonably expect IEEE 754-conformant behaviour). Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work. -- Mark
Hi Mark, Aha, good, you posted an email to python-dev, good :-) Last days, I was trying to collect more data about this topic, especially find platforms which *don't* support IEEE 754, before posting to python-dev. Nowadays, outside museums, it's hard to find computers which don't implement IEEE 754. * Since 1998, IBM S/390 supports IEEE 754. * Compaq/DEC VAX didn't use IEEE 754, but are no longer built since 2000 (you cant still use Python 3.10 if it can be built on it!) * Some Cray computers like Cray SV1 (1998) don't implement IEEE 754, but other Cray computers like Cray T90 (1995) implement IEEE 754. There are embedded devices with no hardware FPU: in this case, the FPU is implemented in software. I expect it to implement IEEE 754. Is CPython a good target for such small CPUs which have no hardware FPU? MicroPython may better fit their needs. On the other side, all moderns CPU architectures support IEEE 754: Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC, Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC. Sources: * https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_p... * https://stackoverflow.com/questions/2234468/do-any-real-world-cpus-not-use-i... On Mon, Feb 7, 2022 at 7:25 PM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
In the past, when we deprecated the support for an old platform, we didn't suddenly remove the code. We made sure that it's no longer possible to build on it. So if anyone notices, it's easy to revert (ex: remove the few lines in configure). Would it make sense to trigger a build error on plaforms which don't support IEEE 754 in Python 3.11, and later decide if it's time to remove the code in in Python 3.12? Well. If you ask *me*, I'm in favor of removing the code right now. If someone needs to support a platform which doesn't support IEEE 754, the support can be maintained *outside* the Python source code, as external patches or as a Git fork, no? Honestly, I never got access to any machine which doesn't support IEEE 754 (or nobody told me!).
- Should we require IEEE 754 floating-point for Python-the-language?
Right now, I have no opinion on this question.
Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.
My comment on bpo-46656: "Python uses Py_NAN without "#ifdef Py_NAN" guard since 2008. Building Python with Py_NO_NAN never worked. Nobody reported such build failure in the last 14 years..." If anyone would try building/using Python on a platform without NAN, I would expect that we would get a bug report or an email. I'm not aware of anything like that, so it seems like nobody uses Python on such platform. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Mon, Feb 7, 2022 at 11:06 AM Victor Stinner <vstinner@python.org> wrote:
Hi Mark,
Aha, good, you posted an email to python-dev, good :-) Last days, I was trying to collect more data about this topic, especially find platforms which *don't* support IEEE 754, before posting to python-dev.
Nowadays, outside museums, it's hard to find computers which don't implement IEEE 754.
* Since 1998, IBM S/390 supports IEEE 754. * Compaq/DEC VAX didn't use IEEE 754, but are no longer built since 2000 (you cant still use Python 3.10 if it can be built on it!) * Some Cray computers like Cray SV1 (1998) don't implement IEEE 754, but other Cray computers like Cray T90 (1995) implement IEEE 754.
There are embedded devices with no hardware FPU: in this case, the FPU is implemented in software. I expect it to implement IEEE 754. Is CPython a good target for such small CPUs which have no hardware FPU? MicroPython may better fit their needs.
On the other side, all moderns CPU architectures support IEEE 754: Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC, Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC.
Sources:
* https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_p... * https://stackoverflow.com/questions/2234468/do-any-real-world-cpus-not-use-i...
On Mon, Feb 7, 2022 at 7:25 PM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
In the past, when we deprecated the support for an old platform, we didn't suddenly remove the code. We made sure that it's no longer possible to build on it. So if anyone notices, it's easy to revert (ex: remove the few lines in configure).
Would it make sense to trigger a build error on plaforms which don't support IEEE 754 in Python 3.11, and later decide if it's time to remove the code in in Python 3.12?
Well. If you ask *me*, I'm in favor of removing the code right now. If someone needs to support a platform which doesn't support IEEE 754, the support can be maintained *outside* the Python source code, as external patches or as a Git fork, no?
Honestly, I never got access to any machine which doesn't support IEEE 754 (or nobody told me!).
- Should we require IEEE 754 floating-point for Python-the-language?
Right now, I have no opinion on this question.
Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.
My comment on bpo-46656: "Python uses Py_NAN without "#ifdef Py_NAN" guard since 2008. Building Python with Py_NO_NAN never worked. Nobody reported such build failure in the last 14 years..."
If anyone would try building/using Python on a platform without NAN, I would expect that we would get a bug report or an email. I'm not aware of anything like that, so it seems like nobody uses Python on such platform.
Victor
It's 2022. We should just require both NaN and IEEE-754. By the time a system is large enough to build and run CPython, it is exceedingly unlikely that it will not be able to do that. Even if it means software emulation of IEEE-754 floating point on something large yet odd enough not to have a FPU. The places where non-IEEE-754 and non-NaN floating point are likely to exist are in specialized parallelized coprocessor hardware. Not the general purpose CPU running CPython. -gps
On Mon, Feb 7, 2022 at 12:31 PM Gregory P. Smith <greg@krypto.org> wrote:
On Mon, Feb 7, 2022 at 11:06 AM Victor Stinner <vstinner@python.org> wrote:
Hi Mark,
Aha, good, you posted an email to python-dev, good :-) Last days, I was trying to collect more data about this topic, especially find platforms which *don't* support IEEE 754, before posting to python-dev.
Nowadays, outside museums, it's hard to find computers which don't implement IEEE 754.
* Since 1998, IBM S/390 supports IEEE 754. * Compaq/DEC VAX didn't use IEEE 754, but are no longer built since 2000 (you cant still use Python 3.10 if it can be built on it!) * Some Cray computers like Cray SV1 (1998) don't implement IEEE 754, but other Cray computers like Cray T90 (1995) implement IEEE 754.
There are embedded devices with no hardware FPU: in this case, the FPU is implemented in software. I expect it to implement IEEE 754. Is CPython a good target for such small CPUs which have no hardware FPU? MicroPython may better fit their needs.
On the other side, all moderns CPU architectures support IEEE 754: Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC, Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC.
Sources:
* https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_p... * https://stackoverflow.com/questions/2234468/do-any-real-world-cpus-not-use-i...
On Mon, Feb 7, 2022 at 7:25 PM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
In the past, when we deprecated the support for an old platform, we didn't suddenly remove the code. We made sure that it's no longer possible to build on it. So if anyone notices, it's easy to revert (ex: remove the few lines in configure).
Would it make sense to trigger a build error on plaforms which don't support IEEE 754 in Python 3.11, and later decide if it's time to remove the code in in Python 3.12?
Well. If you ask *me*, I'm in favor of removing the code right now. If someone needs to support a platform which doesn't support IEEE 754, the support can be maintained *outside* the Python source code, as external patches or as a Git fork, no?
Honestly, I never got access to any machine which doesn't support IEEE 754 (or nobody told me!).
- Should we require IEEE 754 floating-point for Python-the-language?
Right now, I have no opinion on this question.
Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.
My comment on bpo-46656: "Python uses Py_NAN without "#ifdef Py_NAN" guard since 2008. Building Python with Py_NO_NAN never worked. Nobody reported such build failure in the last 14 years..."
If anyone would try building/using Python on a platform without NAN, I would expect that we would get a bug report or an email. I'm not aware of anything like that, so it seems like nobody uses Python on such platform.
Victor
It's 2022. We should just require both NaN and IEEE-754.
By the time a system is large enough to build and run CPython, it is exceedingly unlikely that it will not be able to do that. Even if it means software emulation of IEEE-754 floating point on something large yet odd enough not to have a FPU.
The places where non-IEEE-754 and non-NaN floating point are likely to exist are in specialized parallelized coprocessor hardware. Not the general purpose CPU running CPython.
Do we have a buildbot that has a CPU or OS that can't handle IEEE-754? What are the chances we will get one? If the answers are "none" and "slim", then it seems reasonable to require NaN and IEEE-754.
On Tue, Feb 8, 2022 at 12:19 AM Brett Cannon <brett@python.org> wrote:
Do we have a buildbot that has a CPU or OS that can't handle IEEE-754? What are the chances we will get one? If the answers are "none" and "slim", then it seems reasonable to require NaN and IEEE-754.
All CPU architectures of current buildbot workers use IEEE 754 (little or big endian). I looked at the following CPU architectures: * x86 (Intel 32-bit) * x86-64 (Intel 64-bit) * armv7l (ARM 32-bit) * aarch64 (ARM 64-bit) * ppc64, ppc64le (PowerPC 64-bit) * s390x (IBM) There is also a "SPARCv9 Oracle Solaris 11.4" worker, but sadly it seems to be disconnected for 3 months. The SPARCv9 architecture uses IEEE 754. On buildbots, I checked float.__getformat__("double"). It is logged as "builtins.float.double_format" in the "pythoninfo" step of buidbots. Little-endian examples: * x86-64 (AMD64 Arch Linux Asan Debug 3.x): IEEE, little-endian * x86 (Gentoo Non-Debug with X 3.x): IEEE, little-endian * aarch64 (CentOS9 LTO + PGO 3.10): IEEE, little-endian * aarch64 (Fedora Rawhide Clang 3.x): IEEE, little-endian * aarch64: MacOS M1 (ARM64 macOS 3.x): IEEE, little-endian * aarch64 (ARM64 Windows 3.x): IEEE, little-endian * PPC64LE (CentOS9 3.10): IEEE, little-endian * armv7l (ARM Raspbian 3.x): IEEE, little-endian Big-endian examples: * PPC64 (AIX 3.x): IEEE, big-endian * s390x (Debian 3.x): IEEE, big-endian float.__getformat__("double") is initialized by _PyFloat_InitState() at Python startup with this code: --- #if SIZEOF_DOUBLE == 8 { double x = 9006104071832581.0; if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) detected_double_format = ieee_big_endian_format; else if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) detected_double_format = ieee_little_endian_format; else detected_double_format = unknown_format; } #else detected_double_format = unknown_format; #endif --- Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Mon, Feb 7, 2022 at 11:02 AM Victor Stinner <vstinner@python.org> wrote:
Hi Mark,
Aha, good, you posted an email to python-dev, good :-) Last days, I was trying to collect more data about this topic, especially find platforms which *don't* support IEEE 754, before posting to python-dev.
Nowadays, outside museums, it's hard to find computers which don't implement IEEE 754.
* Since 1998, IBM S/390 supports IEEE 754. * Compaq/DEC VAX didn't use IEEE 754, but are no longer built since 2000 (you cant still use Python 3.10 if it can be built on it!) * Some Cray computers like Cray SV1 (1998) don't implement IEEE 754, but other Cray computers like Cray T90 (1995) implement IEEE 754.
There are embedded devices with no hardware FPU: in this case, the FPU is implemented in software. I expect it to implement IEEE 754. Is CPython a good target for such small CPUs which have no hardware FPU? MicroPython may better fit their needs.
On the other side, all moderns CPU architectures support IEEE 754: Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC, Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC.
RISC-V also supports the 2008 IEEE-754 spec: https://riscv.org/technical/specifications/ . -Brett
Sources:
* https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_p... * https://stackoverflow.com/questions/2234468/do-any-real-world-cpus-not-use-i...
On Mon, Feb 7, 2022 at 7:25 PM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
In the past, when we deprecated the support for an old platform, we didn't suddenly remove the code. We made sure that it's no longer possible to build on it. So if anyone notices, it's easy to revert (ex: remove the few lines in configure).
Would it make sense to trigger a build error on plaforms which don't support IEEE 754 in Python 3.11, and later decide if it's time to remove the code in in Python 3.12?
Well. If you ask *me*, I'm in favor of removing the code right now. If someone needs to support a platform which doesn't support IEEE 754, the support can be maintained *outside* the Python source code, as external patches or as a Git fork, no?
Honestly, I never got access to any machine which doesn't support IEEE 754 (or nobody told me!).
- Should we require IEEE 754 floating-point for Python-the-language?
Right now, I have no opinion on this question.
Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.
My comment on bpo-46656: "Python uses Py_NAN without "#ifdef Py_NAN" guard since 2008. Building Python with Py_NO_NAN never worked. Nobody reported such build failure in the last 14 years..."
If anyone would try building/using Python on a platform without NAN, I would expect that we would get a bug report or an email. I'm not aware of anything like that, so it seems like nobody uses Python on such platform.
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/PPVBMCKL... Code of Conduct: http://python.org/psf/codeofconduct/
From the perspective of some that writes a lot of computational code: On Mon, Feb 7, 2022 at 10:25 AM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
Yes, and yes, together, as Mark suggests - Should we require IEEE 754 floating-point for Python-the-language?
I would say no, but it should be recommended -- see Victor's example of Micro Python -- though does anyone have an authority over that? IIUC, Micro Python already has a few differences -- but is anyone saying they shouldn't call it Python? Though, with a quick perusal of teh docs, micro Python does seem to support NaN and Inf, at least -- not sure about the rest of 754. While we're at it are 64 bit floats required for either cPython or Python the language? IIUC, the main platform that this was an issue for, for computationally heavy code, was Cray -- but it seems they're 754 compatible now, yes? Thanks for doing this, -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Mon, Feb 7, 2022 at 4:52 PM Christopher Barker <pythonchb@gmail.com> wrote:
From the perspective of some that writes a lot of computational code:
On Mon, Feb 7, 2022 at 10:25 AM Mark Dickinson <dickinsm@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation?
Yes, and yes, together, as Mark suggests
- Should we require IEEE 754 floating-point for Python-the-language?
I would say no, but it should be recommended -- see Victor's example of Micro Python -- though does anyone have an authority over that? IIUC, Micro Python already has a few differences -- but is anyone saying they shouldn't call it Python? Though, with a quick perusal of teh docs, micro Python does seem to support NaN and Inf, at least -- not sure about the rest of 754.
PEP 7 and PEP 11 are only about CPython. Other Python VMs full or partial are free to do what they want with how they implement floats if at all. (they're build time optional in MicroPython as are many Python language syntax features) While we're at it are 64 bit floats required for either cPython or Python
the language?
CPython: yes. we use a double. Python the language: no. (float is single precision on many micropython platforms as it saves precious ram and performance, plus microcontroller fpu hardware like an M4 is usually single precision 32bit)
On 2/8/2022 1:35 AM, Gregory P. Smith wrote:
While we're at it are 64 bit floats required for either cPython or Python the language?
CPython: yes. we use a double. Python the language: no. (float is single precision on many micropython platforms as it saves precious ram and performance, plus microcontroller fpu hardware like an M4 is usually single precision 32bit)
Agreed. CPython should be specific, Python should be as vague as possible. Otherwise, we would prevent _by specification_ using Python as a scripting language for things where floats may not even be relevant. It's not even about hardware capabilities (unless you believe Python is only for running in a terminal, in which case we'll just have to disagree). It could be about an app using short Python snippets as a customisation language for some domain-specific task rather than inventing a brand new language for it. The more "it's-only-Python-if-it-has-X" restrictions we have, the less appealing we become. Cheers, Steve
On Tue, Feb 08, 2022 at 12:44:46PM +0000, Steve Dower wrote:
Agreed. CPython should be specific, Python should be as vague as possible. Otherwise, we would prevent _by specification_ using Python as a scripting language for things where floats may not even be relevant.
I don't think that being *vague* is desirable. Being *permissive* on the other hand may be.
It's not even about hardware capabilities (unless you believe Python is only for running in a terminal, in which case we'll just have to disagree). It could be about an app using short Python snippets as a customisation language for some domain-specific task rather than inventing a brand new language for it.
I'm sorry, I don't understand what point you are trying to make here. Regarding "running in a terminal", I don't see how the environment (terminal, GUI, something else) is relevent. Regarding your second point, about domain-specific tasks, I don't understand. If your domain-specific task doesn't need to use floats, just don't use floats. There's surely no need to invent a whole new language for some task because your app only needs to do string processing, say. Have I misunderstood what you are trying to say? -- Steve
On 2/8/2022 9:54 PM, Steven D'Aprano wrote:
Regarding your second point, about domain-specific tasks, I don't understand. If your domain-specific task doesn't need to use floats, just don't use floats. There's surely no need to invent a whole new language for some task because your app only needs to do string processing, say. Have I misunderstood what you are trying to say?
Only that some people get very concerned about following "standards", and we can help them by making our standard easier to follow ("more permissive," as you suggest, is fine wording by me). Some people see things that are written down as rules to be followed, while others see them as guidelines. When we write something down wrong or loosely, we can cause a surprising amount of angst within the first group (for example, I fairly frequently get worried questions about the VS version for building CPython due to how the wiki and devguide mention it). I'm just trying to make sure we don't add more of these, by ensuring that we properly scope things. In this case *CPython requires IEEE 754 floats*, but the Python language does not. Cheers, Steve
On Mon, Feb 07, 2022 at 05:35:17PM -0800, Gregory P. Smith wrote:
CPython: yes. we use a double. Python the language: no. (float is single precision on many micropython platforms as it saves precious ram and performance, plus microcontroller fpu hardware like an M4 is usually single precision 32bit)
If we are to *officially* support non-double floats, it would be nice if sys.float_info were to tell us explicitly how wide the floats are rather than having to try to reverse engineer it from the other information there. A floating point expert can probably look at this: sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1) and immediately recognise that those values imply a 64-bit float, but I expect most people will not. If Python the language is going to support single, double, quad precision floats, and maybe even minifloats with just 16 or even fewer bits, then can we please add a field to float_info telling us how many bits the floats have? -- Steve
On Tue, Feb 8, 2022 at 2:25 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Feb 07, 2022 at 05:35:17PM -0800, Gregory P. Smith wrote:
CPython: yes. we use a double. Python the language: no. (float is single precision on many micropython platforms as it saves precious ram and performance, plus microcontroller fpu hardware like an M4 is usually single precision 32bit)
If we are to *officially* support non-double floats, it would be nice if sys.float_info were to tell us explicitly how wide the floats are rather than having to try to reverse engineer it from the other information there.
A floating point expert can probably look at this:
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16, radix=2, rounds=1)
and immediately recognise that those values imply a 64-bit float, but I expect most people will not. If Python the language is going to support single, double, quad precision floats, and maybe even minifloats with just 16 or even fewer bits, then can we please add a field to float_info telling us how many bits the floats have?
There is no need to know how many bits it is. The meaningful information about precision and accuracy from a math point of view is already expressed in float_info. the size in bits isn't relevant. You can derive the size from that if you'd like and are willing to assume a binary format. binary_bits = ~mant_dig+log2(max_exp)... But that tells you less than sys.float_info. -gps
I think you skimmed over "A floating point expert can probably look at this ... " I remember a time when I just assumed more bits was better, and a later time when I figured smaller was better, and a still later time when I wanted to match the published requirements for bitsize. So that was several years when I didn't really understand the tradeoffs, but could benefit from (or at least write better documentation) knowing the size. During those years, I would have recognized the importance of 1024, but would probably not have bothered interpreting 2.220446049250313. A method (or docstring) with a more friendly interface would be good. -jJ
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation? - Should we require IEEE 754 floating-point for Python-the-language? I don't have strong opinions on the first two, but for the language definition, I think the most we should say is "if an implementation does not support IEEE 754 floating-point, this must be mentioned in the documentation as an implementation limit."
On Tue, 08 Feb 2022 01:12:22 -0000 "Jim J. Jewett" <jimjjewett@gmail.com> wrote:
- Should we require the presence of NaNs in order for CPython to build? - Should we require IEEE 754 floating-point for CPython-the-implementation? - Should we require IEEE 754 floating-point for Python-the-language?
I don't have strong opinions on the first two, but for the language definition, I think the most we should say is "if an implementation does not support IEEE 754 floating-point, this must be mentioned in the documentation as an implementation limit."
This is more about platform support than about implementation support. An implementation using a C "float" or "double" (or whatever the equivalent in another implementation language) will de facto support IEEE 754 on platforms that support it. Sure, a Python implementation may also choose to rewrite its own floating-point type from scratch, but that sounds unlikely. Regards Antoine.
On Mon, Feb 07, 2022 at 06:23:52PM +0000, Mark Dickinson wrote:
- Should we require IEEE 754 floating-point for CPython-the-implementation? - Should we require IEEE 754 floating-point for Python-the-language?
If the answer to those questions are Yes, that rules out using Unums, posits, sigmoid numbers etc as the builtin float. (The terminology is a bit vague, sorry.) Do we want that? https://ieeexplore.ieee.org/document/8080000 https://en.wikipedia.org/wiki/Unum_%28number_format%29 https://github.com/interplanetary-robot/SigmoidNumbers Posits are hyped as "better than IEEE-754", I have no idea if it is all hype or if they actually are better or just different. -- Steve
On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Mon, Feb 07, 2022 at 06:23:52PM +0000, Mark Dickinson wrote:
- Should we require IEEE 754 floating-point for CPython-the-implementation? - Should we require IEEE 754 floating-point for Python-the-language?
If the answer to those questions are Yes, that rules out using Unums, posits, sigmoid numbers etc as the builtin float. (The terminology is a bit vague, sorry.) Do we want that?
It does not rule anything else out should they become viable. This is just a statement that to build cpython we require ieee754 support. It does not say anything about how our Python float type is implemented internally. Should a meaningful large-os platform come along that promotes the use of a different format available from C we could make use of that and loosen the policy as needed. What updating our requirement for CPython would mean is that the likely unexercised outside our own __set_format__ using test suite "unknown_format" code in https://github.com/python/cpython/blob/main/Objects/floatobject.c could go away until such time as a platform with an actual different format springs into existence. Driveby floatobject.c code inspection: It is odd that we do ieee_be/ieee_le/unknown conditionals as a runtime check rather than configure time check as that means we compile the code for three implementations into our float implementation on all platforms despite them each using only one - I guess that was done for testing purposes presumably in the 1.x era when viable platforms were weirder as standards traction grew - today I'd call that dead code bloat. -gps
https://ieeexplore.ieee.org/document/8080000
https://en.wikipedia.org/wiki/Unum_%28number_format%29
https://github.com/interplanetary-robot/SigmoidNumbers
Posits are hyped as "better than IEEE-754", I have no idea if it is all hype or if they actually are better or just different.
-- Steve _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7LUAQ32Z... Code of Conduct: http://python.org/psf/codeofconduct/
On Tue, Feb 08, 2022 at 05:48:46PM -0800, Gregory P. Smith wrote:
On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
If the answer to those questions are Yes, that rules out using Unums, posits, sigmoid numbers etc as the builtin float. (The terminology is a bit vague, sorry.) Do we want that?
It does not rule anything else out should they become viable. This is just a statement that to build cpython we require ieee754 support. It does not say anything about how our Python float type is implemented internally.
Posits do not implement IEEE-754. They aren't merely a different internal representation of the IEEE-754 floating point standard, they implement a different numeric system altogether. Things that IEEE-754 require, such as signed zero and signed infinity, are not supported by posits. Posits include a single unsigned zero and a single unsigned infinity. Earlier, I made a mistake: I misremembered that posits support a single NAN, and so I removed Mark's question about requiring NANs from my quoting. I was wrong to do so: posits do not have any NANs. So if we require NANs, or IEEE-754, that rules out posits as the builtin float. -- Steve
On Tue, Feb 8, 2022 at 10:20 PM Steven D'Aprano <steve@pearwood.info> wrote:
On Tue, Feb 08, 2022 at 05:48:46PM -0800, Gregory P. Smith wrote:
On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <steve@pearwood.info> wrote:
If the answer to those questions are Yes, that rules out using Unums, posits, sigmoid numbers etc as the builtin float. (The terminology is a bit vague, sorry.) Do we want that?
It does not rule anything else out should they become viable. This is just a statement that to build cpython we require ieee754 support. It does not say anything about how our Python float type is implemented internally.
Posits do not implement IEEE-754. They aren't merely a different internal representation of the IEEE-754 floating point standard, they implement a different numeric system altogether.
Things that IEEE-754 require, such as signed zero and signed infinity, are not supported by posits. Posits include a single unsigned zero and a single unsigned infinity.
Earlier, I made a mistake: I misremembered that posits support a single NAN, and so I removed Mark's question about requiring NANs from my quoting. I was wrong to do so: posits do not have any NANs.
So if we require NANs, or IEEE-754, that rules out posits as the builtin float.
As Greg said, all we are discussing is requiring NaN and IEEE-754 in order to compile CPython. There's nothing saying we can't change how things are implemented in CPython in the future if something else made sense. And if someone wants to fork CPython to implement something else they can as well. And none of this has any bearing on the language based on the trend of this conversation's answer to Mark's third question. So I am not seeing how any decision we make from this thread about CPython and IEEE-754 rules anything out in the future?
The consensus is to require IEEE 754 to build CPython, but not require it in the Python language specification. Updates (changed merged in bpo-46656): * Building Python 3.11 now requires a C11 compiler without optional C11 features. I wrote it in What's New in Python 3.11 and the PEP 7. * Building Python 3.11 now requires support for floating point Not-a-Number (NaN): remove the Py_NO_NAN macro. Victor
On Mon, Feb 7, 2022 at 5:11 PM Victor Stinner vstinner@python.org wrote:
I made a change to require C99 <math.h> "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11. What if it was built with MSVC which is usually just C90?
On Mon, Feb 7, 2022 at 9:12 AM Victor Stinner <vstinner@python.org> wrote:
Hi,
I made a change to require C99 <math.h> "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11.
Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7: https://www.python.org/dev/peps/pep-0007/
I modified Python 3.11 to require more C99 features of the <math.h> header:
* bpo-45440: copysign(), hypot(), isfinite(), isinf(), isnan(), round() * bpo-46640: NAN
After my NAN change (bpo-46640), Petr Viktorin asked me to update the PEP 7. I proposed a change to simply say that "Python 3.11 and newer versions use C99": https://github.com/python/peps/pull/2309
I would prefer to not have to give an exhaustive list of C99 features used by CPython, since it's unclear to me what belongs to C99 or to ISO C89. As I wrote before, Python already uses C99 features since Python 3.6.
On my PEP PR, Guido van Rossum asked me to escalate the discussion to python-dev, so here I am :-)
In "C99", the number "99" refers to the year 1999, the standard is now 23 years old: https://en.wikipedia.org/wiki/C99
In 2022, C99 is now well supported by C compilers supported by Python: GCC, clang, MSVC.
I think if those compilers fully C99 at this point we should consider just moving completely over to C99. -Brett
I don't know if AIX XLC supports C99. AIX provides a "c99" compiler compatible with C99. It also seems like GCC is usable on AIX.
I don't know if ICC supports C99. Python doesn't officially the ICC compiler, the ICC buildbots are gone a few years ago. But sometimes I make some changes to enhance the ICC support, when the change is small enough.
Note: Python also uses C11 <stdatomic.h>, but it's not required: there are fallbacks for compilers which don't support it.
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J5FSP6J4... Code of Conduct: http://python.org/psf/codeofconduct/
https://www.ibm.com/docs/en/SSGH3R_16.1.0/pdf/getstart.pdf As far as reading page 3, xlclang fully supports C89/C99/C11. xlc fully supports C89/C99, and partially supports C11. On Tue, Feb 8, 2022 at 8:57 AM Brett Cannon <brett@python.org> wrote:
On Mon, Feb 7, 2022 at 9:12 AM Victor Stinner <vstinner@python.org> wrote:
Hi,
I made a change to require C99 <math.h> "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11.
Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7: https://www.python.org/dev/peps/pep-0007/
I modified Python 3.11 to require more C99 features of the <math.h> header:
* bpo-45440: copysign(), hypot(), isfinite(), isinf(), isnan(), round() * bpo-46640: NAN
After my NAN change (bpo-46640), Petr Viktorin asked me to update the PEP 7. I proposed a change to simply say that "Python 3.11 and newer versions use C99": https://github.com/python/peps/pull/2309
I would prefer to not have to give an exhaustive list of C99 features used by CPython, since it's unclear to me what belongs to C99 or to ISO C89. As I wrote before, Python already uses C99 features since Python 3.6.
On my PEP PR, Guido van Rossum asked me to escalate the discussion to python-dev, so here I am :-)
In "C99", the number "99" refers to the year 1999, the standard is now 23 years old: https://en.wikipedia.org/wiki/C99
In 2022, C99 is now well supported by C compilers supported by Python: GCC, clang, MSVC.
I think if those compilers fully C99 at this point we should consider just moving completely over to C99.
-Brett
I don't know if AIX XLC supports C99. AIX provides a "c99" compiler compatible with C99. It also seems like GCC is usable on AIX.
I don't know if ICC supports C99. Python doesn't officially the ICC compiler, the ICC buildbots are gone a few years ago. But sometimes I make some changes to enhance the ICC support, when the change is small enough.
Note: Python also uses C11 <stdatomic.h>, but it's not required: there are fallbacks for compilers which don't support it.
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/J5FSP6J4... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZLDOBJUV... Code of Conduct: http://python.org/psf/codeofconduct/
-- Inada Naoki <songofacandy@gmail.com>
07.02.22 19:09, Victor Stinner пише:
After my NAN change (bpo-46640), Petr Viktorin asked me to update the PEP 7. I proposed a change to simply say that "Python 3.11 and newer versions use C99":
But public headers should be compatible with C++, and not all C99 features are compatible with C++. It is not a matter of style, it is a matter of compatibility.
On 07. 02. 22 18:09, Victor Stinner wrote:
Hi,
I made a change to require C99 <math.h> "NAN" constant and I'm was asked to update the PEP 7 to clarify the C subset is needed to build Python 3.11.
Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7: https://www.python.org/dev/peps/pep-0007/
I modified Python 3.11 to require more C99 features of the <math.h> header:
* bpo-45440: copysign(), hypot(), isfinite(), isinf(), isnan(), round() * bpo-46640: NAN
After my NAN change (bpo-46640), Petr Viktorin asked me to update the PEP 7. I proposed a change to simply say that "Python 3.11 and newer versions use C99": https://github.com/python/peps/pull/2309
I would prefer to not have to give an exhaustive list of C99 features used by CPython, since it's unclear to me what belongs to C99 or to ISO C89. As I wrote before, Python already uses C99 features since Python 3.6.
On my PEP PR, Guido van Rossum asked me to escalate the discussion to python-dev, so here I am :-)
In "C99", the number "99" refers to the year 1999, the standard is now 23 years old: https://en.wikipedia.org/wiki/C99
In 2022, C99 is now well supported by C compilers supported by Python: GCC, clang, MSVC.
Does MSVC support all of C99? I haven't found any documentation claiming that... but I'm also not familiar with MSVC. Do we need to support a subset like "C99 except the features that were removed in C11"?
On 2/8/2022 9:52 AM, Petr Viktorin wrote:
On 07. 02. 22 18:09, Victor Stinner wrote:
In 2022, C99 is now well supported by C compilers supported by Python: GCC, clang, MSVC.
Does MSVC support all of C99? I haven't found any documentation claiming that... but I'm also not familiar with MSVC.
Do we need to support a subset like "C99 except the features that were removed in C11"?
I couldn't find any information either (from public sources, at least, haven't asked the team directly yet). All the C99 library is supposedly supported, but there are (big?) gaps in the compiler support. Possibly these are features that were removed in C11? I don't know what is on that list. It seems C11 and C17 are supported [1], though I know for sure we have contributors who aren't using a recent enough compiler version to have it :) Personally, I see no reason to "require" C99 as a whole. We have a style guide already, and can list the additional compiler features that we allow along with guidance on updating existing code and ensuring compatibility. I don't see much risk requiring the C99 standard library, though. It's the compiler features that seem to have less coverage. Cheers, Steve [1]: https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving...
On Tue, Feb 8, 2022 at 2:02 PM Steve Dower <steve.dower@python.org> wrote:
All the C99 library is supposedly supported, but there are (big?) gaps in the compiler support.
Some Visual Studio 2019 updates on the Microsoft blog. March 2020: C99 _Pragma https://devblogs.microsoft.com/cppblog/announcing-full-support-for-a-c-c-con...
The _Pragma operator has been one of the long-standing deficiencies of the preprocessor, and a blocker in being standard conformant in C++ and C99.
September 2020: C11 and C17 https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving... Maybe a more practical approach would be to use C99 "except of features not supported by MSVC of Visual Studio 2019"? In practice, we can try to support VS 2017, the version currently recommended by the devguide: https://devguide.python.org/setup/#windows-compiling We already have fallbacks for compilers which don't support C99. We don't have to remove these fallbacks if they are still needed by popular C compilers. For example, I'm fine with keeping basic compatbility support with ICC if it remains reasonable, even if we don't "officially" support ICC (ex: no buildbot). Victor -- Night gathers, and now my watch begins. It shall not end until my death.
Maybe a more practical approach would be to use C99 "except of features not supported by MSVC of Visual Studio 2019"?
This could be formulated in a more neutral way by saying "C99 without the things that became optional in C11", or perhaps "C11 without optional features" (at least from the POV of MSVC, haven't checked the other compilers/platforms for C11-compliance).
In practice, we can try to support VS 2017, the version currently recommended by the devguide:
That is becoming dated quickly, as Microsoft has deprecated, and is removing, that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see: https://github.com/actions/virtual-environments/issues/4312. It's understandable in the sense that they don't want to support a third version in addition to vs2022 and vs2019, but the net effect is that very few (open source) projects will keep using vs2017 going forward.
On 09. 02. 22 4:39, h.vetinari@gmx.com wrote:
Maybe a more practical approach would be to use C99 "except of features not supported by MSVC of Visual Studio 2019"?
This could be formulated in a more neutral way by saying
"C99 without the things that became optional in C11", or perhaps
That sounds like a better wording!
"C11 without optional features" (at least from the POV of MSVC, haven't checked the other compilers/platforms for C11-compliance).
That's an interesting idea -- what's keeping us from C11? In other words: the main thing keeping us from C99 is MSVC support, and since that compiler apparently skipped C99, should we skip it as well?
In practice, we can try to support VS 2017, the version currently recommended by the devguide:
That is becoming dated quickly, as Microsoft has deprecated, and is removing, that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see: https://github.com/actions/virtual-environments/issues/4312.
It's understandable in the sense that they don't want to support a third version in addition to vs2022 and vs2019, but the net effect is that very few (open source) projects will keep using vs2017 going forward. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IIZ6LXK2... Code of Conduct: http://python.org/psf/codeofconduct/
On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <encukou@gmail.com> wrote:
On 09. 02. 22 4:39, h.vetinari@gmx.com wrote:
Maybe a more practical approach would be to use C99 "except of features not supported by MSVC of Visual Studio 2019"?
This could be formulated in a more neutral way by saying
"C99 without the things that became optional in C11", or perhaps
That sounds like a better wording!
"C11 without optional features" (at least from the POV of MSVC, haven't checked the other compilers/platforms for C11-compliance).
That's an interesting idea -- what's keeping us from C11?
No one asking before, probably because we have been trying to get to C99 for so long. 😄
In other words: the main thing keeping us from C99 is MSVC support, and since that compiler apparently skipped C99, should we skip it as well?
If we think "C11 without optional features" is widely supported then I think that's a fine target to have. For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_feature... . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me. -Brett
In practice, we can try to support VS 2017, the version currently recommended by the devguide:
That is becoming dated quickly, as Microsoft has deprecated, and is removing, that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see: https://github.com/actions/virtual-environments/issues/4312.
It's understandable in the sense that they don't want to support a third version in addition to vs2022 and vs2019, but the net effect is that very few (open source) projects will keep using vs2017 going forward. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IIZ6LXK2... Code of Conduct: http://python.org/psf/codeofconduct/
Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ERQQFXNA... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon <brett@python.org> wrote:
On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <encukou@gmail.com> wrote:
On 09. 02. 22 4:39, h.vetinari@gmx.com wrote:
That's an interesting idea -- what's keeping us from C11?
No one asking before, probably because we have been trying to get to C99 for so long. 😄
In other words: the main thing keeping us from C99 is MSVC support, and since that compiler apparently skipped C99, should we skip it as well?
If we think "C11 without optional features" is widely supported then I think that's a fine target to have.
For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_feature... . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me.
I like it. I want to use anonymous union. It makes complex structure like PyDictKeysObject simple a little. I confirmed that XLC supports it. https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-u... Regards, -- Inada Naoki <songofacandy@gmail.com>
On 10. 02. 22 0:30, Inada Naoki wrote:
On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon <brett@python.org> wrote:
On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <encukou@gmail.com> wrote:
On 09. 02. 22 4:39, h.vetinari@gmx.com wrote:
That's an interesting idea -- what's keeping us from C11?
No one asking before, probably because we have been trying to get to C99 for so long. 😄
In other words: the main thing keeping us from C99 is MSVC support, and since that compiler apparently skipped C99, should we skip it as well?
If we think "C11 without optional features" is widely supported then I think that's a fine target to have.
For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_feature... . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me.
I like it. I want to use anonymous union. It makes complex structure like PyDictKeysObject simple a little.
I confirmed that XLC supports it. https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-u...
Ah, I've also wanted anonymous unions in the past! There's a little problem in that they're not valid in C++, so we can't have them in public headers. We'll need to mention C++ if we update the standard.
On Thu, Feb 10, 2022 at 10:28 AM Petr Viktorin <encukou@gmail.com> wrote:
Ah, I've also wanted anonymous unions in the past! There's a little problem in that they're not valid in C++, so we can't have them in public headers.
We'll need to mention C++ if we update the standard.
IMO we only have to care about C++ in Include/ header files: make sure that "Python.h" is usable in C++. Outside this directory, we don't have to care about C++ compatibility. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Thu, Feb 10, 2022 at 6:31 PM Petr Viktorin <encukou@gmail.com> wrote:
I like it. I want to use anonymous union. It makes complex structure like PyDictKeysObject simple a little.
I confirmed that XLC supports it. https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-u...
Ah, I've also wanted anonymous unions in the past! There's a little problem in that they're not valid in C++, so we can't have them in public headers.
C++ 11 supports anonymous union with some reasonable limitations. https://en.cppreference.com/w/cpp/language/union XL C/C++ also support it. So we can use it if we decided to use it. Regards, -- Inada Naoki <songofacandy@gmail.com>
IMO we need to distinguish the public C API which should be as much compatible as possible, target the oldest C standard, and the Python internals can require a more recent C standard. For example, today maybe it's reasonable to requires C99 for Include/ headers (public .h files) and support C11 for Python internals (private .c files). Victor
On Wed, 09 Feb 2022 03:39:42 -0000 h.vetinari@gmx.com wrote:
That is becoming dated quickly, as Microsoft has deprecated, and is removing, that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see: https://github.com/actions/virtual-environments/issues/4312.
It's understandable in the sense that they don't want to support a third version in addition to vs2022 and vs2019, but the net effect is that very few (open source) projects will keep using vs2017 going forward.
Not every project using CPython is an open source project, though. And CPython probably supports more platforms than GHA or Azure do. Regards Antoine.
According to the docs[^1], MSVC does _not_ support all of C99. OTOH they claim conformance with both C11 and C17, using the /std:c11 and /std:c17 compiler flags. [^1]: https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-st...
On 8 Feb 2022, at 10:52, Petr Viktorin <encukou@gmail.com> wrote:
Does MSVC support all of C99? I haven't found any documentation claiming that... but I'm also not familiar with MSVC.
Hi, I updated my PEP 7 PR to use C99 in the public C API and "a subset of" C11 in Python internals: "Python 3.11 and newer versions use C99 in the public C API and use a subset of C11 in Python internals. The public C API should be compatible with C++. The C11 subset are features supported by GCC 8.5, clang 8.0, and MSVC of Visual Studio 2017." https://github.com/python/peps/pull/2309/files GCC 8.5 is the version chosen by RHEL 8. It should provide C11 features that we care about. I pickled clang 8.0 because it's had been released in 2019 and so should be available on most operating systems. FreeBSD uses clang by default. FreeBSD 13 uses clang 11. And it seems like we still care about support Visual Studio 2017, even if Visual Studio 2019 and 2022 are available. I chose to not require supporting AIX XLC. Inada-san wrote: "xlclang fully supports C89/C99/C11. xlc fully supports C89/C99, and partially supports C11." I guess that in practice, we can test a PR on buildbots when trying "new shiny" C11 feature. Moreover, if a C11 feature is missing, it's usually not too complicated to use a workaround for C99 and older. Victor
And it seems like we still care about support Visual Studio 2017, even if Visual Studio 2019 and 2022 are available.
Can we make this more concrete? Do we know of affected parties? Numbers of affected users? Or are we just conservatively guesstimating the thickness of the long tail of support? On top of that, I think the formulation you chose does not map correctly to actual compiler capabilities. C99 (minus C11 optionals) only arrived in VS2019, there were still gaps in VS2017. I would advocate bumping the requirement to VS2019. Yes, there is a built-in reluctance to update toolchains, but MSFT has been _extremely_ conservative w.r.t. ABI-compatibility (e.g. the whole story with [[msvc::no_unique_address]]), and I just don't see the argument why a non-negligible portion of users cannot upgrade to the fully-compatible-with-everything-previously VS2019.
"Python 3.11 and newer versions use C99 in the public C API and use a subset of C11 in Python internals. The public C API should be compatible with C++. The C11 subset are features supported by GCC 8.5, clang 8.0, and MSVC of Visual Studio 2017."
If we were to require VS2019, then the formulation could be much nicer (IMO): """ Python 3.11 and newer versions use C99 in the public C API (without those features that became optional in C11), while Python internals may use C11 (again without optional features). The public C API should be compatible with C++. """
Ok, let me try something simpler: "Python 3.11 and newer versions use C11 without optional features. The public C API should be compatible with C++." https://github.com/python/peps/pull/2309/files Victor
On 24 Feb 2022, at 11:45, Victor Stinner <vstinner@python.org> wrote:
Ok, let me try something simpler:
"Python 3.11 and newer versions use C11 without optional features. The public C API should be compatible with C++." https://github.com/python/peps/pull/2309/files
Should is often read as meaning optional when writing specs. Can you say “must be compatible with C++”. Barry
Victor _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C32XCK5W... Code of Conduct: http://python.org/psf/codeofconduct/
On Thu, Feb 24, 2022 at 11:10 PM Barry <barry@barrys-emacs.org> wrote:
"Python 3.11 and newer versions use C11 without optional features. The public C API should be compatible with C++." https://github.com/python/peps/pull/2309/files
Should is often read as meaning optional when writing specs. Can you say “must be compatible with C++”.
I plan to attempt to write an actual test for that, rather than a vague sentence in a PEP. For now, "should" is a deliberate choice: I don't know exactly which C++ version should be targeted and if it's really an issue or not. For example, C++20 reserves the "module" keyword, whereas Python uses it in its C API. Example: PyAPI_FUNC(int) PyModule_AddType(PyObject *module, PyTypeObject *type); See: * https://bugs.python.org/issue39355 * https://github.com/pythoncapi/pythoncapi_compat/issues/21 -- I made a change in the datatable project to add Python 3.11 support using the pythoncapi_compat.h header file. Problem: this *C* header file produced new warnings in datatable extension module which with built with a C++ compiler: https://github.com/h2oai/datatable/pull/3231#issuecomment-1032864790 Examples: | src/core/lib/pythoncapi_compat.h:272:52: warning: zero as null pointer constant [-Wzero-as-null-pointer-constant] | || tstate->c_profilefunc != NULL); | ^~~~ | nullptr and | src/core/lib/pythoncapi_compat.h:170:12: warning: use of old-style cast [-Wold-style-cast] | return (PyCodeObject *)_Py_StealRef(PyFrame_GetCode(frame)); | ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ I made pythoncapi_compat.h compatible with C++ (fix C++ compiler warnings) by using nullptr and reinterpret_cast<TYPE>(EXPR) cast if the __cplusplus macro is defined, or NULL and ((TYPE)(EXPR)) cast otherwise. datatable also uses #include "Python.h". I don't know there were only C++ compiler warnings on "pythoncapi_compat.h". Maybe because datatable only uses static inline functions from "pythoncapi_compat.h", but it may also emit the same warnings if tomorrow some static inline functions of "Python.h" are used. For now, I prefer to put a reminder in PEP 7 that the "Python.h" C API is consumed by C++ projects. Victor -- Night gathers, and now my watch begins. It shall not end until my death.
On Thu, Feb 24, 2022 at 3:27 PM Victor Stinner <vstinner@python.org> wrote:
On Thu, Feb 24, 2022 at 11:10 PM Barry <barry@barrys-emacs.org> wrote:
"Python 3.11 and newer versions use C11 without optional features. The public C API should be compatible with C++." https://github.com/python/peps/pull/2309/files
Should is often read as meaning optional when writing specs. Can you say “must be compatible with C++”.
I plan to attempt to write an actual test for that, rather than a vague sentence in a PEP. For now, "should" is a deliberate choice: I don't know exactly which C++ version should be targeted and if it's really an issue or not.
Agreed. "should" is good because we're not even clear if we currently actually comply with C++ standards. i.e. https://bugs.python.org/issue40120 suggests we technically may not for C++ (it is not strictly a superset of C as we all like to pretend), though for practical purposes regardless of standards compilers tend to allow that. We're likely overspecifying in any document we create about what we require because the only definition any of us are actually capable of making for what we require is "does it compile with this compiler on this platform? If yes, then we appear to support it. can we guarantee that? only with buildbots or other CI" - We're generally not versed in specific language standards (aside from compiler folks, who is?), and compilers don't comply strictly with all the shapes of those anyways for either practical or hysterical reasons. So no matter what we claim to aspire to, reality is always murkier. A document about requirements is primarily useful to give guidance to what we expect to be aligned with and what is or isn't allowed to be used in new code. Our code itself always has the final say. -gps
For example, C++20 reserves the "module" keyword, whereas Python uses it in its C API. Example:
PyAPI_FUNC(int) PyModule_AddType(PyObject *module, PyTypeObject *type);
See:
* https://bugs.python.org/issue39355 * https://github.com/pythoncapi/pythoncapi_compat/issues/21
--
I made a change in the datatable project to add Python 3.11 support using the pythoncapi_compat.h header file. Problem: this *C* header file produced new warnings in datatable extension module which with built with a C++ compiler: https://github.com/h2oai/datatable/pull/3231#issuecomment-1032864790
Examples:
| src/core/lib/pythoncapi_compat.h:272:52: warning: zero as null pointer constant [-Wzero-as-null-pointer-constant] | || tstate->c_profilefunc != NULL); | ^~~~ | nullptr
and
| src/core/lib/pythoncapi_compat.h:170:12: warning: use of old-style cast [-Wold-style-cast] | return (PyCodeObject *)_Py_StealRef(PyFrame_GetCode(frame)); | ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
I made pythoncapi_compat.h compatible with C++ (fix C++ compiler warnings) by using nullptr and reinterpret_cast<TYPE>(EXPR) cast if the __cplusplus macro is defined, or NULL and ((TYPE)(EXPR)) cast otherwise.
datatable also uses #include "Python.h". I don't know there were only C++ compiler warnings on "pythoncapi_compat.h". Maybe because datatable only uses static inline functions from "pythoncapi_compat.h", but it may also emit the same warnings if tomorrow some static inline functions of "Python.h" are used.
For now, I prefer to put a reminder in PEP 7 that the "Python.h" C API is consumed by C++ projects.
Victor -- Night gathers, and now my watch begins. It shall not end until my death. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/RGNBM5CS... Code of Conduct: http://python.org/psf/codeofconduct/
On 23Feb2022 2240, h.vetinari@gmx.com wrote:
On top of that, I think the formulation you chose does not map correctly to actual compiler capabilities. C99 (minus C11 optionals) only arrived in VS2019, there were still gaps in VS2017.
Additionally, Visual Studio version doesn't directly map to the compiler version. For example, VS2022 contains multiple older versions of the compiler. Currently we use the v142 toolset for releases (selected by default in desktop development workloads in VS 2019), and can't migrate to v143 (selected by default in VS 2022) until some compiler bugs are fixed. If we're going to specify anything about MSVC, I'd prefer it was the v142 label rather than the Visual Studio version (though we can include the latter as a convenience for the reader, and hopefully clearly enough that people stop filing bug reports about it...) Cheers, Steve
participants (16)
-
Antoine Pitrou -
Barry -
Brett Cannon -
Christopher Barker -
Erlend Aasland -
Gregory P. Smith -
h.vetinari＠gmx.com -
Inada Naoki -
Jeremiah Vivian -
Jim J. Jewett -
Mark Dickinson -
Petr Viktorin -
Serhiy Storchaka -
Steve Dower -
Steven D'Aprano -
Victor Stinner