more-precise instructions for "Python.h first"?

Boost.Python is now trying hard to accomodate the "Python.h before system headers rule". Unfortunately, we still need a wrapper around Python.h, at least for some versions of Python, so that we can work around some issues like: // // Python's LongObject.h helpfully #defines ULONGLONG_MAX for us // even when it's not defined by the system which confuses Boost's // config // To cope with that correctly, we need to see <limits.h> (a system header) before longobject.h. Currently, we're including <limits.h>, then <patchlevel.h>, well, and then the wrapper gets a little complicated adjusting for various compilers. Anyway, the point is that I'd like to have the rule changed to "You have to include Python.h or xxxx.h before any system header" where xxxx.h is one of the other existing headers #included in Python.h that is responsible for setting up whatever macros cause this inclusion-order requirement in the first place (preferably not LongObject.h!) That way I might be able to get those configuration issues sorted out without violating the #inclusion order rule. What I have now seems to work, but I'd rather do the right thing (TM). -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams <dave@boost-consulting.com> writes:
If I understand correctly, you want to follow the rule "I want to change things as long it continues to work for me". For that, you don't need any permission. If it works for you, you can ignore any rules you feel uncomfortable with. The rule is there for people who don't want to understand the specific details of system configuration. If you manage to get a consistent configuration in a different way, just go for it. You should make sure then that your users can't run into problems, though. Regards, Martin

martin@v.loewis.de (Martin v. Löwis) writes:
Then you don't understand correctly.
I can't make sure that my users can't run into problems without understanding everything about Python and Posix which causes the rule to exist in the first place (and I don't), and continuously monitoring Python into the future to make sure that the distribution of Posix configuration information across its headers doesn't change in a way that invalidates previous assumptions. The current rule doesn't work for me, but I'd like to be following _some_ sanctioned rule to reduce the chance of problems today and in the future. I'm making an educated guess that the rule is much more-sweeping than Python development needs it to be. Isn't there some Python internal configuration header which can be #included first and which will accomplish all the same things as far as system-header inclusion order is concerned? -- Dave Abrahams Boost Consulting www.boost-consulting.com

[David Abrahams]
Well, we're in the same boat: we can't ensure that Python users can't run into problems without knowing everything about how the union of all flaky platform headers (not just POSIX) will work for all time either. We do know that, historically, we've been able to hack around all problems that have arisen by sticking Python.h first. There's no guarantee that will always work, though.
The current rule doesn't work for me,
It doesn't work for us either, at least not on principle. It just happens to have "been enough" in practice so far.
Sorry, I don't know. I do know that changes to the headers are usually followed by problems on some handful of systems that can't be predicted in advance. I'll note that life is much easier if you drop support for all systems other than Win32 <wink>.

David Abrahams <dave@boost-consulting.com> writes:
I can't think of a less strict rule that is still comprehensible. I did not comprehend the wording that you first proposed, since it referred to preconditions that are fuzzy (what is xxxx.h?) If you come up with a wording of a less strict rule, I can try to guess whether that rule would also work, on the systems I care about. In drafting such wording, keep in mind that the primary motivation for this rule is LFS support, where Python wants to select the 64-bit API (aka "large file") on systems that support that. So any system header that could potentially be affected by LFS selection can only be included after LFS selection has been done. Regards, Martin

martin@v.loewis.de (Martin v. Löwis) writes:
Fabulous. I'd like the rule changed from: "Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included." to: "Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h or pyconfig.h (Python's configuration header), before any standard headers are included." That is precise wording, taken from http://www.python.org/doc/current/api/includes.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams <dave@boost-consulting.com> writes:
Ok. I think this rule may break on SGI, where Python.h has #if defined(__sgi) && defined(WITH_THREAD) && !defined(_SGI_MP_SOURCE) #define _SGI_MP_SOURCE #endif Now, I have never seen an SGI machine in my life, and I have no idea what _SGI_MP_SOURCE is. However it seems that it is important that this is defined before stdio.h is included (or perhaps it was important at some point in time). Apart from that, I can't find any problems with that rule (but there may be problems that I'm overlooking). I recommend that you follow this rule, anyway; if you discover any problems by doing so, please let us know. Regards, Martin

On Monday, Jun 2, 2003, at 07:41 Europe/Amsterdam, Martin v. Löwis wrote:
This define seems to handle how concurrent access and locking of stdio data structures is handled (and, incidentally, also how errno is declared in errno.h). So it does indeed seem important that this is in scope before including errno.h or stdio.h. It should be easy enough to have configure stick the _SGI_MP_SOURCE definition in pyconfig.h, if someone is willing to do the work. For me Pyhon stopped compiling on Irix a while ago, and I can't be bothered to look into it. -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

David Abrahams <dave@boost-consulting.com> writes:
If I understand correctly, you want to follow the rule "I want to change things as long it continues to work for me". For that, you don't need any permission. If it works for you, you can ignore any rules you feel uncomfortable with. The rule is there for people who don't want to understand the specific details of system configuration. If you manage to get a consistent configuration in a different way, just go for it. You should make sure then that your users can't run into problems, though. Regards, Martin

martin@v.loewis.de (Martin v. Löwis) writes:
Then you don't understand correctly.
I can't make sure that my users can't run into problems without understanding everything about Python and Posix which causes the rule to exist in the first place (and I don't), and continuously monitoring Python into the future to make sure that the distribution of Posix configuration information across its headers doesn't change in a way that invalidates previous assumptions. The current rule doesn't work for me, but I'd like to be following _some_ sanctioned rule to reduce the chance of problems today and in the future. I'm making an educated guess that the rule is much more-sweeping than Python development needs it to be. Isn't there some Python internal configuration header which can be #included first and which will accomplish all the same things as far as system-header inclusion order is concerned? -- Dave Abrahams Boost Consulting www.boost-consulting.com

[David Abrahams]
Well, we're in the same boat: we can't ensure that Python users can't run into problems without knowing everything about how the union of all flaky platform headers (not just POSIX) will work for all time either. We do know that, historically, we've been able to hack around all problems that have arisen by sticking Python.h first. There's no guarantee that will always work, though.
The current rule doesn't work for me,
It doesn't work for us either, at least not on principle. It just happens to have "been enough" in practice so far.
Sorry, I don't know. I do know that changes to the headers are usually followed by problems on some handful of systems that can't be predicted in advance. I'll note that life is much easier if you drop support for all systems other than Win32 <wink>.

David Abrahams <dave@boost-consulting.com> writes:
I can't think of a less strict rule that is still comprehensible. I did not comprehend the wording that you first proposed, since it referred to preconditions that are fuzzy (what is xxxx.h?) If you come up with a wording of a less strict rule, I can try to guess whether that rule would also work, on the systems I care about. In drafting such wording, keep in mind that the primary motivation for this rule is LFS support, where Python wants to select the 64-bit API (aka "large file") on systems that support that. So any system header that could potentially be affected by LFS selection can only be included after LFS selection has been done. Regards, Martin

martin@v.loewis.de (Martin v. Löwis) writes:
Fabulous. I'd like the rule changed from: "Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h before any standard headers are included." to: "Since Python may define some pre-processor definitions which affect the standard headers on some systems, you must include Python.h or pyconfig.h (Python's configuration header), before any standard headers are included." That is precise wording, taken from http://www.python.org/doc/current/api/includes.html -- Dave Abrahams Boost Consulting www.boost-consulting.com

David Abrahams <dave@boost-consulting.com> writes:
Ok. I think this rule may break on SGI, where Python.h has #if defined(__sgi) && defined(WITH_THREAD) && !defined(_SGI_MP_SOURCE) #define _SGI_MP_SOURCE #endif Now, I have never seen an SGI machine in my life, and I have no idea what _SGI_MP_SOURCE is. However it seems that it is important that this is defined before stdio.h is included (or perhaps it was important at some point in time). Apart from that, I can't find any problems with that rule (but there may be problems that I'm overlooking). I recommend that you follow this rule, anyway; if you discover any problems by doing so, please let us know. Regards, Martin

On Monday, Jun 2, 2003, at 07:41 Europe/Amsterdam, Martin v. Löwis wrote:
This define seems to handle how concurrent access and locking of stdio data structures is handled (and, incidentally, also how errno is declared in errno.h). So it does indeed seem important that this is in scope before including errno.h or stdio.h. It should be easy enough to have configure stick the _SGI_MP_SOURCE definition in pyconfig.h, if someone is willing to do the work. For me Pyhon stopped compiling on Irix a while ago, and I can't be bothered to look into it. -- Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman
participants (4)
-
David Abrahams
-
Jack Jansen
-
martin@v.loewis.de
-
Tim Peters