what environment variable should contain compiler warning suppression flags?
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in. I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility. So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Brett Cannon wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
It is not important to me as flags set to BASECFLAGS, CFLAGS, OPT or EXTRA_CFLAGS will set makefile macros CFLAGS and after distribution python distutil will use them to build extension modules. So all variable are equal for builds. Also after configure without OPT variable set we could check what script select for build platform and to rerun configure with OPT+own_flags set on command line (! ;) ) . Roumen
On Wed, Jun 23, 2010 at 14:53, Brett Cannon <brett@python.org> wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
Brett Cannon wrote:
On Wed, Jun 23, 2010 at 14:53, Brett Cannon <brett@python.org> wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
I think you need to come up with a different solution and revert the change... OPT has historically been the only variable to use for adjusting the Python C compiler settings. As the name implies this was usually used to adjust the optimizer settings, including raising the optimization level from the default or disabling it. With your change CFLAGS will always override OPT and thus any optimization definitions made in OPT will no longer have an effect. Note that CFLAGS defines -O2 on many platforms. In your particular case, you should try setting OPT to "... -Wno-unused-value ..." (ie. replace -Wall with your setting). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 27 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 21 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Sat, Jun 26, 2010 at 16:37, M.-A. Lemburg <mal@egenix.com> wrote:
Brett Cannon wrote:
On Wed, Jun 23, 2010 at 14:53, Brett Cannon <brett@python.org> wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
I think you need to come up with a different solution and revert the change...
OPT has historically been the only variable to use for adjusting the Python C compiler settings.
Just found the relevant section in the README.
As the name implies this was usually used to adjust the optimizer settings, including raising the optimization level from the default or disabling it.
It meant optional to me, not optimization. I hate abbreviations sometimes.
With your change CFLAGS will always override OPT and thus any optimization definitions made in OPT will no longer have an effect.
That was the point; OPT defines defaults through configure.in and I simply wanted to add to those instead of having OPT completely overwritten by me.
Note that CFLAGS defines -O2 on many platforms.
So then wouldn't that mean they want that to be the optimization level? Or is the historical reason that default exists is so that some default exists but to expect the application to override as desired?
In your particular case, you should try setting OPT to "... -Wno-unused-value ..." (ie. replace -Wall with your setting).
So what is CFLAGS for then? ``configure -h`` says it's for "C compiler flags"; that's extremely ambiguous. And it doesn't help that OPT is not mentioned by ``configure -h`` as that is what I have always gone by to know what flags are available for compilation. -Brett
-- Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jun 27 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 21 days to go
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
Brett Cannon wrote:
On Sat, Jun 26, 2010 at 16:37, M.-A. Lemburg<mal@egenix.com> wrote:
Brett Cannon wrote:
On Wed, Jun 23, 2010 at 14:53, Brett Cannon<brett@python.org> wrote: [SKIP] Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
Right now you cannot change order of CFLAGS and OPT
I think you need to come up with a different solution and revert the change...
OPT has historically been the only variable to use for adjusting the Python C compiler settings.
Just found the relevant section in the README.
As the name implies this was usually used to adjust the optimizer settings, including raising the optimization level from the default or disabling it.
It meant optional to me, not optimization. I hate abbreviations sometimes.
With your change CFLAGS will always override OPT and thus any optimization definitions made in OPT will no longer have an effect.
That was the point; OPT defines defaults through configure.in and I simply wanted to add to those instead of having OPT completely overwritten by me.
Now if you confirm that (see configure.in ) : # Optimization messes up debuggers, so turn it off for # debug builds. OPT="-g -O0 -Wall $STRICT_PROTO" is not issue for py3k then left you commit as is (Note that Mark point this). But if optimization "messes up debuggers" you may revert change. I know that is difficult to reach consensus on compiler/preprocessor flags for python build process. Next is a shot list with issues about this: - "Python 2.5 64 bit compile fails on Solaris 10/gcc 4.1.1" : http://bugs.python.org/issue1628484 (committed/rejected) - "Python does not honor "CFLAGS" environment variable" : http://bugs.python.org/issue1453 (wont fix) - "configure: allow user-provided CFLAGS to override AC_PROG_CC defaults" : http://bugs.python.org/issue8211 (fixed) This is still open "configure doesn't set up CFLAGS properly" ( http://bugs.python.org/issue1104249 ) - must be closed as fixed.
Note that CFLAGS defines -O2 on many platforms.
So then wouldn't that mean they want that to be the optimization level? Or is the historical reason that default exists is so that some default exists but to expect the application to override as desired?
In your particular case, you should try setting OPT to "... -Wno-unused-value ..." (ie. replace -Wall with your setting).
So what is CFLAGS for then? ``configure -h`` says it's for "C compiler flags"; that's extremely ambiguous. And it doesn't help that OPT is not mentioned by ``configure -h`` as that is what I have always gone by to know what flags are available for compilation.
-Brett
If you like to see some flags the could you look into http://bugs.python.org/issue3718 how to define an option to be visible by configure --help. In addition AC_ARG_VAR will allow environment variable to be cached for subsequent run of config.status otherwise you must specify only on configure command line. About all XXflags variables if is good configure script to be simplified to use only CPPFLAGS and CFLAGS to minimize configuration troubles and other build falures. A good sample if configure set preprocessor/compiler flags other then CPPFLAGS/CFLAGS is this issue "OSX: duplicate -arch flags in CFLAGS breaks sysconfig" ( http://bugs.python.org/issue8607 ) Roumen
On Sat, Jun 26, 2010 at 4:37 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Brett Cannon wrote:
On Wed, Jun 23, 2010 at 14:53, Brett Cannon <brett@python.org> wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
I think you need to come up with a different solution and revert the change...
OPT has historically been the only variable to use for adjusting the Python C compiler settings.
As the name implies this was usually used to adjust the optimizer settings, including raising the optimization level from the default or disabling it.
With your change CFLAGS will always override OPT and thus any optimization definitions made in OPT will no longer have an effect.
Note that CFLAGS defines -O2 on many platforms.
In your particular case, you should try setting OPT to "... -Wno-unused-value ..." (ie. replace -Wall with your setting).
The python configure environment variables are really confused. If OPT is intended to be user-overridden for optimization settings, it shouldn't be used to set -Wall and -Wstrict-prototypes. If it's intended to set warning options, it shouldn't also set optimization options. Setting the user-visible customization option on the configure command line shouldn't stomp unrelated defaults. In configure-based systems, CFLAGS is traditionally (http://sources.redhat.com/automake/automake.html#Flag-Variables-Ordering) the way to tack options onto the end of the command line. Python breaks this by threading flags through CFLAGS in the makefile, which means they all get stomped if the user sets CFLAGS on the make command line. We should instead use another spelling ("CFlags"?) for the internal variable, and append $(CFLAGS) to it. AC_PROG_CC is the macro that sets CFLAGS to -g -O2 on gcc-based systems (http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html#index-AC_0...). If Python's configure.in sets an otherwise-empty CFLAGS to -g before calling AC_PROG_CC, AC_PROG_CC won't change it. Or we could just preserve the users CFLAGS setting across AC_PROG_CC regardless of whether it's set, to let the user set CFLAGS on the configure line without stomping any defaults.
On Sun, Jun 27, 2010 at 6:46 AM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
AC_PROG_CC is the macro that sets CFLAGS to -g -O2 on gcc-based systems (http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html#index-AC_0...). If Python's configure.in sets an otherwise-empty CFLAGS to -g before calling AC_PROG_CC, AC_PROG_CC won't change it. Or we could just preserve the users CFLAGS setting across AC_PROG_CC regardless of whether it's set, to let the user set CFLAGS on the configure line without stomping any defaults.
I think saving and restoring CFLAGS across AC_PROG_CC was attempted in http://bugs.python.org/issue8211 . It turned out that it broke OS X universal builds. I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary? Mark
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
On Sun, Jun 27, 2010 at 6:46 AM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
AC_PROG_CC is the macro that sets CFLAGS to -g -O2 on gcc-based systems (http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html#index-AC_0...). If Python's configure.in sets an otherwise-empty CFLAGS to -g before calling AC_PROG_CC, AC_PROG_CC won't change it. Or we could just preserve the users CFLAGS setting across AC_PROG_CC regardless of whether it's set, to let the user set CFLAGS on the configure line without stomping any defaults.
I think saving and restoring CFLAGS across AC_PROG_CC was attempted in http://bugs.python.org/issue8211 . It turned out that it broke OS X universal builds.
Thanks for the link to the issue. http://bugs.python.org/issue8366 says Ronald Oussoren fixed the universal builds without reverting the CFLAGS propagation.
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful). Jeffrey
On Sun, Jun 27, 2010 at 13:37, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
On Sun, Jun 27, 2010 at 6:46 AM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
AC_PROG_CC is the macro that sets CFLAGS to -g -O2 on gcc-based systems (http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html#index-AC_0...). If Python's configure.in sets an otherwise-empty CFLAGS to -g before calling AC_PROG_CC, AC_PROG_CC won't change it. Or we could just preserve the users CFLAGS setting across AC_PROG_CC regardless of whether it's set, to let the user set CFLAGS on the configure line without stomping any defaults.
I think saving and restoring CFLAGS across AC_PROG_CC was attempted in http://bugs.python.org/issue8211 . It turned out that it broke OS X universal builds.
Thanks for the link to the issue. http://bugs.python.org/issue8366 says Ronald Oussoren fixed the universal builds without reverting the CFLAGS propagation.
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful).
I went ahead and reverted the change, but it does seem like the build environment could use a cleanup.
Brett Cannon wrote:
On Sun, Jun 27, 2010 at 13:37, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
On Sun, Jun 27, 2010 at 6:46 AM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
AC_PROG_CC is the macro that sets CFLAGS to -g -O2 on gcc-based systems (http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html#index-AC_0...). If Python's configure.in sets an otherwise-empty CFLAGS to -g before calling AC_PROG_CC, AC_PROG_CC won't change it. Or we could just preserve the users CFLAGS setting across AC_PROG_CC regardless of whether it's set, to let the user set CFLAGS on the configure line without stomping any defaults.
I think saving and restoring CFLAGS across AC_PROG_CC was attempted in http://bugs.python.org/issue8211 . It turned out that it broke OS X universal builds.
Thanks for the link to the issue. http://bugs.python.org/issue8366 says Ronald Oussoren fixed the universal builds without reverting the CFLAGS propagation.
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful).
Please see the discussion on the ticket for details. AC_PROG_CC provides the basic defaults for the CFLAGS compiler settings depending on which compiler is chosen/found: http://www.gnu.org/software/hello/manual/autoconf/C-Compiler.html
I went ahead and reverted the change, but it does seem like the build environment could use a cleanup.
Thanks and, indeed, the build system environment variable usage does need a cleanup. It's a larger project, though, and one that will likely break existing build setups. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 28 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 20 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Mon, Jun 28, 2010 at 12:38 PM, M.-A. Lemburg <mal@egenix.com> wrote:
On Sun, Jun 27, 2010 at 13:37, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful).
Please see the discussion on the ticket for details.
Yes, I've done that. It's repeatedly asserted in that discussion that AC_PROG_CC should be allowed to initialize an otherwise empty CFLAGS, but nowhere in that discussion does it explain *why* this is desirable. What would be so bad about not allowing AC_PROG_CC to initialize CFLAGS? (E.g., by setting an otherwise empty CFLAGS to '-g' before the AC_PROG_CC invocation.) That would fix the issue of the unwanted -O2 flag that AC_PROG_CC otherwise adds. Mark
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 12:38 PM, M.-A. Lemburg <mal@egenix.com> wrote:
On Sun, Jun 27, 2010 at 13:37, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful).
Please see the discussion on the ticket for details.
Yes, I've done that. It's repeatedly asserted in that discussion that AC_PROG_CC should be allowed to initialize an otherwise empty CFLAGS, but nowhere in that discussion does it explain *why* this is desirable. What would be so bad about not allowing AC_PROG_CC to initialize CFLAGS? (E.g., by setting an otherwise empty CFLAGS to '-g' before the AC_PROG_CC invocation.) That would fix the issue of the unwanted -O2 flag that AC_PROG_CC otherwise adds.
Why do you think that the default -O2 is unwanted and how do you know whether the compiler accepts -g as option ? -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 28 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 20 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Mon, Jun 28, 2010 at 3:04 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Why do you think that the default -O2 is unwanted
Because it can cause debug builds of Python to be built with optimization enabled, as we've already seen at least twice.
and how do you know whether the compiler accepts -g as option ?
I don't. It could easily be tested for, though. Alternatively, setting an empty CFLAGS to '-g' could be done just for gcc, since this is the only compiler for which AC_PROG_CC adds -O2. Mark
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 3:04 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Why do you think that the default -O2 is unwanted
Because it can cause debug builds of Python to be built with optimization enabled, as we've already seen at least twice.
Then let me put it this way: How many Python users will compile Python in debug mode ? The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about. I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var. Also note that in some cases you may actually want to have a debug build with optimizations turned on, e.g. to track down a compiler optimization bug.
and how do you know whether the compiler accepts -g as option ?
I don't. It could easily be tested for, though. Alternatively, setting an empty CFLAGS to '-g' could be done just for gcc, since this is the only compiler for which AC_PROG_CC adds -O2.
... and then end up with default Python builds which don't have debug symbols available to track down core dumps, etc. ? AC_PROG_CC checks whether the compiler supports -g and always uses it in that case. The option is supported by more compilers than just GCC. E.g. IBM's xlC and Intel's icl compilers support that option as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 28 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 20 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
M.-A. Lemburg wrote:
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 3:04 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Why do you think that the default -O2 is unwanted
Because it can cause debug builds of Python to be built with optimization enabled, as we've already seen at least twice.
Then let me put it this way:
How many Python users will compile Python in debug mode ?
The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about.
I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var.
Also note that in some cases you may actually want to have a debug build with optimizations turned on, e.g. to track down a compiler optimization bug.
and how do you know whether the compiler accepts -g as option ?
I don't. It could easily be tested for, though. Alternatively, setting an empty CFLAGS to '-g' could be done just for gcc, since this is the only compiler for which AC_PROG_CC adds -O2.
... and then end up with default Python builds which don't have debug symbols available to track down core dumps, etc. ?
AC_PROG_CC checks whether the compiler supports -g and always uses it in that case. The option is supported by more compilers than just GCC. E.g. IBM's xlC and Intel's icl compilers support that option as well.
Sorry, Intel's compiler is called "icc", not "icl": http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us... IBM's compiler: http://publib.boulder.ibm.com/infocenter/macxhelp/v6v81/index.jsp?topic=/com... -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 28 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 20 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Mon, Jun 28, 2010 at 4:28 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 3:04 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Why do you think that the default -O2 is unwanted
Because it can cause debug builds of Python to be built with optimization enabled, as we've already seen at least twice.
Then let me put it this way:
How many Python users will compile Python in debug mode ?
The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about.
I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var.
Shrug. Clearly someone at some point in the past thought it was a good idea to have --with-pydebug builds use -O0. If there's going to be a deliberate decision to drop that now, then that's fine with me.
I don't. It could easily be tested for, though. Alternatively, setting an empty CFLAGS to '-g' could be done just for gcc, since this is the only compiler for which AC_PROG_CC adds -O2.
... and then end up with default Python builds which don't have debug symbols available to track down core dumps, etc. ?
No, I don't see how that follows. I was suggesting that *for gcc only*, an empty CFLAGS be set to '-g' before calling AC_PROG_CC. The *only* effect this would have would be that for gcc, if the user hasn't specified CFLAGS, then CFLAGS ends up being '-g' rather than '-g -O2' after the AC_PROG_CC call. But I'm really not looking for an argument here; I just wanted to understand why you thought AC_PROG_CC setting CFLAGS was important, and you've explained that. Thanks. Mark
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 4:28 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Mark Dickinson wrote:
On Mon, Jun 28, 2010 at 3:04 PM, M.-A. Lemburg <mal@egenix.com> wrote:
Why do you think that the default -O2 is unwanted
Because it can cause debug builds of Python to be built with optimization enabled, as we've already seen at least twice.
Then let me put it this way:
How many Python users will compile Python in debug mode ?
The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about.
I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var.
Shrug. Clearly someone at some point in the past thought it was a good idea to have --with-pydebug builds use -O0. If there's going to be a deliberate decision to drop that now, then that's fine with me.
Ah right, the time machine again :-) OPT already uses -O0 if --with-pydebug is used and the compiler supports -g. Since OPT gets added after CFLAGS, the override already happens...
I don't. It could easily be tested for, though. Alternatively, setting an empty CFLAGS to '-g' could be done just for gcc, since this is the only compiler for which AC_PROG_CC adds -O2.
... and then end up with default Python builds which don't have debug symbols available to track down core dumps, etc. ?
No, I don't see how that follows. I was suggesting that *for gcc only*, an empty CFLAGS be set to '-g' before calling AC_PROG_CC. The *only* effect this would have would be that for gcc, if the user hasn't specified CFLAGS, then CFLAGS ends up being '-g' rather than '-g -O2' after the AC_PROG_CC call. But I'm really not looking for an argument here; I just wanted to understand why you thought AC_PROG_CC setting CFLAGS was important, and you've explained that. Thanks.
Sorry, that was a misunderstand on my part. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 28 2010)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2010-07-19: EuroPython 2010, Birmingham, UK 20 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Jun 28, 2010, at 06:03 PM, M.-A. Lemburg wrote:
OPT already uses -O0 if --with-pydebug is used and the compiler supports -g. Since OPT gets added after CFLAGS, the override already happens...
So nobody's proposing to drop that? Good! Ignore my last message then. :) -Barry
On Jun 28, 2010, at 05:28 PM, M.-A. Lemburg wrote:
How many Python users will compile Python in debug mode ?
How many Python users compile Python at all? :)
The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about.
Sure.
I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var.
Yes, but it's a PITA for several reasons, IMO: * It's pretty underdocumented * It's obscure * It's hard to remember the exact fu needed because you do it infrequently * I usually only remember my mistake when gdb acts funny I strongly suggest that --with-pydebug should be all you need to ensure the best debugging environment, which means turning off compiler optimization. Last time I tried, the -O0 was added and it worked well. (I know this has been in flux though.)
Also note that in some cases you may actually want to have a debug build with optimizations turned on, e.g. to track down a compiler optimization bug.
Yes, but that's *much* more rare than wanting to step through some bit of C code without going crazy. -Barry
Barry Warsaw wrote:
On Jun 28, 2010, at 05:28 PM, M.-A. Lemburg wrote:
How many Python users will compile Python in debug mode ?
How many Python users compile Python at all? :)
The point is that the default build of Python should use the correct production settings for the C compiler out of the box and that's what AC_PROG_CC is all about.
Sure.
I'm pretty sure that Python developers who want to use a debug build have enough code foo to get the -O2 turned into a -O0 either by adjust OPT and/or by providing their own CFLAGS env var.
Yes, but it's a PITA for several reasons, IMO:
* It's pretty underdocumented * It's obscure * It's hard to remember the exact fu needed because you do it infrequently * I usually only remember my mistake when gdb acts funny
I strongly suggest that --with-pydebug should be all you need to ensure the best debugging environment, which means turning off compiler optimization. Last time I tried, the -O0 was added and it worked well. (I know this has been in flux though.)
Also note that in some cases you may actually want to have a debug build with optimizations turned on, e.g. to track down a compiler optimization bug.
Yes, but that's *much* more rare than wanting to step through some bit of C code without going crazy.
I agree - trying to step through -O2 optimized code isn't going to help debug your code, it's going to help you debug the optimizer. That's a very rare use case. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ UPCOMING EVENTS: http://holdenweb.eventbrite.com/ "All I want for my birthday is another birthday" - Ian Dury, 1942-2000
Steve Holden writes:
I agree - trying to step through -O2 optimized code isn't going to help debug your code, it's going to help you debug the optimizer. That's a very rare use case.
Not really. I don't have a lot of practice in debugging at that level, so take it with a grain of salt, but what I've found with XEmacs code is that debugging at -O0 is less often helpful than debugging at -O2. Quite often a naive compilation strategy is used which basically turns those C statements into macros for the underlying assembler, and the code works the way the author thinks it should. But his assumptions are invalid, and when optimized it fails. So I guess you can call that "debugging the optimizer" if you like....
On Sun, Jun 27, 2010 at 9:37 PM, Jeffrey Yasskin <jyasskin@gmail.com> wrote:
On Sun, Jun 27, 2010 at 1:04 PM, Mark Dickinson <dickinsm@gmail.com> wrote:
I think saving and restoring CFLAGS across AC_PROG_CC was attempted in http://bugs.python.org/issue8211 . It turned out that it broke OS X universal builds.
Thanks for the link to the issue. http://bugs.python.org/issue8366 says Ronald Oussoren fixed the universal builds without reverting the CFLAGS propagation.
Yes, you're right (of course). Thanks. Looking at the current configure.in, CFLAGS *does* get saved and restored across the AC_PROG_CC call if it's non-empty; I'm not sure whether this actually (currently) has any effect, since as I understand the documentation CFLAGS won't be touched by AC_PROG_CC if it's already set.
I'm not sure I understand the importance of allowing AC_PROG_CC to set CFLAGS (if CFLAGS is undefined at the point of the AC_PROG_CC); can someone give an example of why this is necessary?
Marc-Andre's argument seems to be "it's possible that AC_PROG_CC adds other flags as well (it currently doesn't, but that may well change in future versions of autoconf)." That seems a little weak to constrain fixing actual problems today. If it ever adds more arguments, we'll need to inspect them anyway to see if they're more like -g or -O2 (wanted or harmful).
Okay; thanks for the explanation. Mark
On Sun, Jun 27, 2010 at 12:37 AM, M.-A. Lemburg <mal@egenix.com> wrote:
Brett Cannon wrote:
On Wed, Jun 23, 2010 at 14:53, Brett Cannon <brett@python.org> wrote:
I finally realized why clang has not been silencing its warnings about unused return values: I have -Wno-unused-value set in CFLAGS which comes before OPT (which defines -Wall) as set in PY_CFLAGS in Makefile.pre.in.
I could obviously set OPT in my environment, but that would override the default OPT settings Python uses. I could put it in EXTRA_CFLAGS, but the README says that's for stuff that tweak binary compatibility.
So basically what I am asking is what environment variable should I use? If CFLAGS is correct then does anyone have any issues if I change the order of things for PY_CFLAGS in the Makefile so that CFLAGS comes after OPT?
Since no one objected I swapped the order in r82259. In case anyone else uses clang to compile Python, this means that -Wno-unused-value will now work to silence the warning about unused return values that is caused by some macros. Probably using -Wno-empty-body is also good to avoid all the warnings triggered by the UCS4 macros in cjkcodecs.
I think you need to come up with a different solution and revert the change...
Agreed; this needs more thought. For one thing, Brett's change has the result that --with-pydebug builds end up being built with -O2 instead of -O0, which can make debugging (e.g., with gdb) somewhat awkward. Mark
participants (8)
-
Barry Warsaw -
Brett Cannon -
Jeffrey Yasskin -
M.-A. Lemburg -
Mark Dickinson -
Roumen Petrov -
Stephen J. Turnbull -
Steve Holden