Policy around compile-time flags in bugfix releases
Hey there, the release managers noticed Stefan adding a new compile flag and backporting it to (soon to be) 3.7.7 and (eventually to be) 3.8.3.
Context: https://bugs.python.org/issue39794
Should this go in? It does look like a new feature to us.
- Ł and Ned
I don't know. However, it seems possible given the symptoms of the original issue (see ASAN-produced stack trace) that there is a bug when using context vars with foreign threads (in this case, C++-created): https://bugs.python.org/issue39776
Regards
Antoine.
Le 01/03/2020 à 11:30, Łukasz Langa a écrit :
Hey there, the release managers noticed Stefan adding a new compile flag and backporting it to (soon to be) 3.7.7 and (eventually to be) 3.8.3.
Context: https://bugs.python.org/issue39794
Should this go in? It does look like a new feature to us.
- Ł and Ned
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/5... Code of Conduct: https://www.python.org/psf/codeofconduct/
FWIW that is a configure flag, not a flag to the compiler, so I see no problem with this. The default build has not changed, it just exposes another way to build the interpreter. We've done this in the past with things like --enable-optimizations and such flags.
In this case it appears to enable "new" code... though from the looks of the issues around this one, perhaps that is really old code just being readded and made available via a flag? That isn't clear to me at a quick glance on the issues. Just that it used to use TLS, was changed to ContextVar, and this flag allows it to keep using TLS.
-gps
On Sun, Mar 1, 2020 at 2:41 AM Antoine Pitrou <antoine@python.org> wrote:
I don't know. However, it seems possible given the symptoms of the original issue (see ASAN-produced stack trace) that there is a bug when using context vars with foreign threads (in this case, C++-created): https://bugs.python.org/issue39776
Regards
Antoine.
Le 01/03/2020 à 11:30, Łukasz Langa a écrit :
Hey there, the release managers noticed Stefan adding a new compile flag and backporting it to (soon to be) 3.7.7 and (eventually to be) 3.8.3.
Context: https://bugs.python.org/issue39794
Should this go in? It does look like a new feature to us.
- Ł and Ned
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/5... Code of Conduct: https://www.python.org/psf/codeofconduct/
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/T... Code of Conduct: https://www.python.org/psf/codeofconduct/
Enabling that workaround for embedders using threads without coroutines while figuring out the real underlying contextvars issue seems like a reasonable change to make in the maintenance branches.
Cheers, Nick.
On Sun, Mar 01, 2020 at 12:17:30PM -0800, Gregory P. Smith wrote:
FWIW that is a configure flag, not a flag to the compiler, so I see no problem with this. The default build has not changed, it just exposes another way to build the interpreter. We've done this in the past with things like --enable-optimizations and such flags.
In this case it appears to enable "new" code... though from the looks of the issues around this one, perhaps that is really old code just being readded and made available via a flag? That isn't clear to me at a quick glance on the issues. Just that it used to use TLS, was changed to ContextVar, and this flag allows it to keep using TLS.
Yes, that's pretty much it. The code that looks new is the exact TLS context code that was present from 3.3 to 3.6.
This code is cleanly separated from the 3.7 ContextVar code using #ifndef WITH_DECIMAL_CONTEXTVAR.
To verify that the TLS code has not changed, checkout the revision before the ContextVar:
bc4123b0b380edda774b8bff2fa1bcc96453b440
Compare _decimal.c to 3.7, 3.8 and master. That diff is empty between the #ifdefs.
I have verified manually on Linux (3.7, 3.8, 3.9) and Windows (3.8, 3.9) that the default is indeed decimal.HAVE_CONTEXTVAR==True. I don't have a compiler for Windows/3.7 but the diff is the same as for 3.8.
Now why the backport? The issue that prompted the flag is this:
https://bugs.python.org/issue39776
While the exact cause is yet unknown (it can be anything like a bug in the interpreter unrelated to ContextVars), I would like to be able to tell people in the future to compile --without-decimal-contextvar if they have such an issue and need a hotfix for their production 3.7.
Also, due to the size of _decimal.c and the complexity of the context handling code, it is really unpleasant to have large diffs between master and the two previous versions.
I understand that Lukasz and Ned will have extra work if something happens, but I don't see how this could be the case here.
Stefan Krah
participants (5)
-
Antoine Pitrou
-
Gregory P. Smith
-
Nick Coghlan
-
Stefan Krah
-
Łukasz Langa