Re: [Python-Dev] Assign to errno allowed?

[Brad Clements]
Except that errno=0 works fine on any platform with a standard- conforming C implementation, and this isn't even a fuzzy POSIX issue -- it's a requirement of the C standard. We can define piles of macros instead, but I expect that, as the years drag on, people will "forget" to use them.

On 24 Sep 2002 at 12:31, Tim Peters wrote:
I don't argue the point that "this stinks". What are our choices: 1. CE port will always be a distinct branch/other cvs, and require gobs of work by "CE porters" for every new core release, changing all the "errno" statements 2. CE changes will always be a pain in the butt for core developers forced to remember Py_SetErrno() 3. No CE port. (oh, also put "NetWare" in there wherever you see the word CE .. and I suspect some other embedded operating systems running on MMU-less processors that can't virtualize errno and don't have TLS) Surprisingly, there aren't that many modules that reference errno directly. In fact, the math stuff is the worst offender .. ;-) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

On 24 Sep 2002 at 20:46, Fredrik Lundh wrote:
Yes, but still on CE you cannot get a pointer to the thread specific errno. You cannot take it's address. So .. errno is #define errno GetLastError() I wish this were not true. Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

I'm strongly against 2. 5 years from now, CE and NetWare and their limitations will only be a vague memory, but this convention will still cripple the Python source code. --Guido van Rossum (home page: http://www.python.org/~guido/)

On 24 Sep 2002 at 14:51, Guido van Rossum wrote:
No arguments about CE, anyway .. (noting that NetWare has reached it's ten year anniversary ;-) I guess then the best solution is a distinct CVS for "the port to oddball platforms" or, the other option is lots of #ifdefs in the code. The original reason I proposed the macro idea was to eliminate multiple nested #ifdefs.. For example, I had trouble figuring out nested #ifdefs in posixmodule, as generated by Mark in his CE port.. It's awful. By switching to macros for "errno =" I was able to clean up a lot of the #ifdefs If changes for CE (and other errno-less OS's) are to be kept in the core, then we'll either have (from Modules/cpickle.c) #ifndef WINDOWCE errno = 0; #else SetLastError(0); #endif l = strtol(s, &endptr, 0); #ifndef WINDOWSCE if (errno || (*endptr != '\n') || (endptr[1] != '\0')) { #else if (GetLastError() || (*endptr != '\n') || (endptr[1] != '\0')) { #endif /* Hm, maybe we've got something long. Let's try reading it as a Python long object. */ #ifndef WINDOWSCE errno = 0; #else SetLastError(0); #endif --- Or --- Py_SetErrno(0) l = strtol(s, &endptr, 0); if (Py_GetErrno() || (*endptr != '\n') || (endptr[1] != '\0')) { /* Hm, maybe we've got something long. Let's try reading it as a Python long object. */ Py_SetErrno(0); Keeping in mind that adding NetWare or (other embedded OS or BIOS that wants to play) then the #ifdef version gets much worse. The macro version doesn't change. There are approximately 140 references to errno in the Modules directory alone. For the Alpha port of Python 2.2 to CE I changed every one of them (at least for any module that could run on CE, which is just about everything that runs on Win32) I've already said that I've made these changes and am willing to make them all again. Once they're in there, how difficult will it be to keep them? New code that uses errno will fail on subsequent builds on these errno-less platforms, but there will only be a handful of changes needed, rather than hundreds on every release of the core. So .. developers who just write "errno = " in the future won't be penalized, rather the porters to errno-less platforms will just have to convert the expression to macro mode. And that conversion process isn't a hardship if it only has to be done once for any given line of code on any given core release.. But if we (porters) have to change 140 references every single time there's a release.. I could see enthusiasm fading away faster than those errno-less platforms ;-) Clearly you guys know what's best better than I do. My line of reasoning for errno macros was: 1. on most platforms the macros compile away to what you'd write in C anyway 2. I'd generate and submit all the initial patches to the core to switch errno references to macros, leaving the burden of review and checkin to the core team (sorry) but not the burden of finding and changing all errno references 3. But once the patches are in-place, future releases wouldn't require nearly as much effort for re-port's to errno-less platforms because only a few lines would need to be "fixed up" to use macros, and only if the changed code didn't use the macros in the first place. 4. Not using the macros in core or extension source isn't an issue for any platform, except errno-less OS's.. at which time that code gets macro'ized at the time of the port. 5. What this does is reduces effort for future ports to crippled systems, at the expense of many initial changes, whose subsequent maintenence shouldn't (hopefully) be a burden, since ports to crippled systems would maintain the changes. Though I do agree, a future mismash of macro's and direct errno references in the core will be ugly and confusing if that occurs. (sorry this is so long, just want to clearly state my case if I have not already done so) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

[Brad, would you mind limiting your messages to lines of about 72 characters at most?]
Absolutely. *If* you want to tackle this, macros for setting errno are the way to go.
Question. You showed that errno was #defined as a call to the right function. Why don't you leave *getting* errno alone? You talk of 100s of places using errno. But how many places *set* errno?
Brad, nobody said they preferred the #ifdef version over the macro! The question is simply whether to use the macro or keep using errno, in tune with Standard C.
Experience shows that each new release will be broken for your platform anyway unless you actively maintain it as we gear up for a release. Even between alpha or beta releases the code base is likely to change in some subtle way that breaks your release. So you'll have to chase down new uses of errno assignment leading up to each release.
If you maintain a branch that uses the errno macro, you could merge the trunk into that branch each time you feel like synching up with the trunk. That's a mostly mechanical process, certainly less than fixing 100s of errno uses manually each time. Or you could simply maintain a patch in the form of a context diff that patches the 100s of places using errno -- assuming this is mostly in stable code, you'd only have to fix up a handful of new occurrences and places where the patch context has gotten out of sync.
They *would* be penalized, because you have to fix their code, and then they have to test it again, etc. It's yet one more thing to worry about.
That's another problem. Whenever there's a massive "peephole" change like this, there are always a few places that are broken but that no reviewer notices and that don't happen to be tested by the test suite. (After all, errno is only consulted when an error occurs, and some errors are darn hard to provoke.)
I totally understand your case. I just don't like having to avoid something that's legal according to the C Standard because of backward platforms. Sure, there were platform-specific changes for many other minority platforms. But none of then AFAIK required us to change something that's Standard C -- these changes were usually about system calls or filename conventions. And we bend over for Win32 because it's the dominant platform. For handhelds, I expect that WinCE will be replaced by something less broken soon. --Guido van Rossum (home page: http://www.python.org/~guido/)

On 24 Sep 2002 at 21:29, Guido van Rossum wrote:
Question. You showed that errno was #defined as a call to the right function. Why don't you leave *getting* errno alone?
Sorry I forgot to clarify that part. Windows CE 1 and 2 have "errno", but in CE 3.0 they improved the OS by eliminating errno and replacing it with GetLastError() I suspect this was done to allow CE to be embedded on new processor types that would not otherwise be supported.
You talk of 100s of places using errno. But how many places *set* errno?
In the modules dir, grep shows: File cmathmodule.c: Py_SetErrno(0); File cPickle.c: Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); File mathmodule.c: Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); But alas for the GetLastError() issue, this wouldn't be so bad.
I agree. I think this will be the best way to go.
I hadn't considered that aspect of the issue. -- Seems then that creating a new SF project to hold a "derivative work?" of the core is the best way to go, but the only difference in this work is a) using macros for errno b) changing "finally" labels to something else. Anyone have a good suggestion for the name of this proposed project? (or, would it be a branch of the core? I'm sorry, I'm still a cvs virgin) I don't think making it CE specific is correct, since it would also be used for NetWare. (oh, did I say 10 years for NetWare? I meant 19) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

Well, *that* is easily solved in pyport.h: #ifdef ...WINCE... #ifndef errno #define errno GetLastError() #endif #endif Much better than changing every use of errno, isn't it? :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

#ifdef ...WINCE... ^^^^^
I wonder if Microsoft foresaw that abbreviation when they chose the name CE... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Greg Ewing]
Speaking of, for many GNU projects, we needed a pre-processor symbol to represent DJGPP and other ports to MS-DOS and Windows, so someone (most probably me) took `DOS' and `WIN' and concatenated them, yielding: #if DOSWIN ... It went for a few years. One day, we were asked by the FSF to avoid this symbol, as it suggested that DOS may win in some way. At the time, this surely meant a lot of places to change, I do not remember if we complied. The fact is that I never had such a pun in head when I made the symbol... -- François Pinard http://www.iro.umontreal.ca/~pinard

On 24 Sep 2002 at 12:31, Tim Peters wrote:
I don't argue the point that "this stinks". What are our choices: 1. CE port will always be a distinct branch/other cvs, and require gobs of work by "CE porters" for every new core release, changing all the "errno" statements 2. CE changes will always be a pain in the butt for core developers forced to remember Py_SetErrno() 3. No CE port. (oh, also put "NetWare" in there wherever you see the word CE .. and I suspect some other embedded operating systems running on MMU-less processors that can't virtualize errno and don't have TLS) Surprisingly, there aren't that many modules that reference errno directly. In fact, the math stuff is the worst offender .. ;-) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

On 24 Sep 2002 at 20:46, Fredrik Lundh wrote:
Yes, but still on CE you cannot get a pointer to the thread specific errno. You cannot take it's address. So .. errno is #define errno GetLastError() I wish this were not true. Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

I'm strongly against 2. 5 years from now, CE and NetWare and their limitations will only be a vague memory, but this convention will still cripple the Python source code. --Guido van Rossum (home page: http://www.python.org/~guido/)

On 24 Sep 2002 at 14:51, Guido van Rossum wrote:
No arguments about CE, anyway .. (noting that NetWare has reached it's ten year anniversary ;-) I guess then the best solution is a distinct CVS for "the port to oddball platforms" or, the other option is lots of #ifdefs in the code. The original reason I proposed the macro idea was to eliminate multiple nested #ifdefs.. For example, I had trouble figuring out nested #ifdefs in posixmodule, as generated by Mark in his CE port.. It's awful. By switching to macros for "errno =" I was able to clean up a lot of the #ifdefs If changes for CE (and other errno-less OS's) are to be kept in the core, then we'll either have (from Modules/cpickle.c) #ifndef WINDOWCE errno = 0; #else SetLastError(0); #endif l = strtol(s, &endptr, 0); #ifndef WINDOWSCE if (errno || (*endptr != '\n') || (endptr[1] != '\0')) { #else if (GetLastError() || (*endptr != '\n') || (endptr[1] != '\0')) { #endif /* Hm, maybe we've got something long. Let's try reading it as a Python long object. */ #ifndef WINDOWSCE errno = 0; #else SetLastError(0); #endif --- Or --- Py_SetErrno(0) l = strtol(s, &endptr, 0); if (Py_GetErrno() || (*endptr != '\n') || (endptr[1] != '\0')) { /* Hm, maybe we've got something long. Let's try reading it as a Python long object. */ Py_SetErrno(0); Keeping in mind that adding NetWare or (other embedded OS or BIOS that wants to play) then the #ifdef version gets much worse. The macro version doesn't change. There are approximately 140 references to errno in the Modules directory alone. For the Alpha port of Python 2.2 to CE I changed every one of them (at least for any module that could run on CE, which is just about everything that runs on Win32) I've already said that I've made these changes and am willing to make them all again. Once they're in there, how difficult will it be to keep them? New code that uses errno will fail on subsequent builds on these errno-less platforms, but there will only be a handful of changes needed, rather than hundreds on every release of the core. So .. developers who just write "errno = " in the future won't be penalized, rather the porters to errno-less platforms will just have to convert the expression to macro mode. And that conversion process isn't a hardship if it only has to be done once for any given line of code on any given core release.. But if we (porters) have to change 140 references every single time there's a release.. I could see enthusiasm fading away faster than those errno-less platforms ;-) Clearly you guys know what's best better than I do. My line of reasoning for errno macros was: 1. on most platforms the macros compile away to what you'd write in C anyway 2. I'd generate and submit all the initial patches to the core to switch errno references to macros, leaving the burden of review and checkin to the core team (sorry) but not the burden of finding and changing all errno references 3. But once the patches are in-place, future releases wouldn't require nearly as much effort for re-port's to errno-less platforms because only a few lines would need to be "fixed up" to use macros, and only if the changed code didn't use the macros in the first place. 4. Not using the macros in core or extension source isn't an issue for any platform, except errno-less OS's.. at which time that code gets macro'ized at the time of the port. 5. What this does is reduces effort for future ports to crippled systems, at the expense of many initial changes, whose subsequent maintenence shouldn't (hopefully) be a burden, since ports to crippled systems would maintain the changes. Though I do agree, a future mismash of macro's and direct errno references in the core will be ugly and confusing if that occurs. (sorry this is so long, just want to clearly state my case if I have not already done so) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

[Brad, would you mind limiting your messages to lines of about 72 characters at most?]
Absolutely. *If* you want to tackle this, macros for setting errno are the way to go.
Question. You showed that errno was #defined as a call to the right function. Why don't you leave *getting* errno alone? You talk of 100s of places using errno. But how many places *set* errno?
Brad, nobody said they preferred the #ifdef version over the macro! The question is simply whether to use the macro or keep using errno, in tune with Standard C.
Experience shows that each new release will be broken for your platform anyway unless you actively maintain it as we gear up for a release. Even between alpha or beta releases the code base is likely to change in some subtle way that breaks your release. So you'll have to chase down new uses of errno assignment leading up to each release.
If you maintain a branch that uses the errno macro, you could merge the trunk into that branch each time you feel like synching up with the trunk. That's a mostly mechanical process, certainly less than fixing 100s of errno uses manually each time. Or you could simply maintain a patch in the form of a context diff that patches the 100s of places using errno -- assuming this is mostly in stable code, you'd only have to fix up a handful of new occurrences and places where the patch context has gotten out of sync.
They *would* be penalized, because you have to fix their code, and then they have to test it again, etc. It's yet one more thing to worry about.
That's another problem. Whenever there's a massive "peephole" change like this, there are always a few places that are broken but that no reviewer notices and that don't happen to be tested by the test suite. (After all, errno is only consulted when an error occurs, and some errors are darn hard to provoke.)
I totally understand your case. I just don't like having to avoid something that's legal according to the C Standard because of backward platforms. Sure, there were platform-specific changes for many other minority platforms. But none of then AFAIK required us to change something that's Standard C -- these changes were usually about system calls or filename conventions. And we bend over for Win32 because it's the dominant platform. For handhelds, I expect that WinCE will be replaced by something less broken soon. --Guido van Rossum (home page: http://www.python.org/~guido/)

On 24 Sep 2002 at 21:29, Guido van Rossum wrote:
Question. You showed that errno was #defined as a call to the right function. Why don't you leave *getting* errno alone?
Sorry I forgot to clarify that part. Windows CE 1 and 2 have "errno", but in CE 3.0 they improved the OS by eliminating errno and replacing it with GetLastError() I suspect this was done to allow CE to be embedded on new processor types that would not otherwise be supported.
You talk of 100s of places using errno. But how many places *set* errno?
In the modules dir, grep shows: File cmathmodule.c: Py_SetErrno(0); File cPickle.c: Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); File mathmodule.c: Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); Py_SetErrno(0); But alas for the GetLastError() issue, this wouldn't be so bad.
I agree. I think this will be the best way to go.
I hadn't considered that aspect of the issue. -- Seems then that creating a new SF project to hold a "derivative work?" of the core is the best way to go, but the only difference in this work is a) using macros for errno b) changing "finally" labels to something else. Anyone have a good suggestion for the name of this proposed project? (or, would it be a branch of the core? I'm sorry, I'm still a cvs virgin) I don't think making it CE specific is correct, since it would also be used for NetWare. (oh, did I say 10 years for NetWare? I meant 19) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

Well, *that* is easily solved in pyport.h: #ifdef ...WINCE... #ifndef errno #define errno GetLastError() #endif #endif Much better than changing every use of errno, isn't it? :-) --Guido van Rossum (home page: http://www.python.org/~guido/)

#ifdef ...WINCE... ^^^^^
I wonder if Microsoft foresaw that abbreviation when they chose the name CE... Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+

[Greg Ewing]
Speaking of, for many GNU projects, we needed a pre-processor symbol to represent DJGPP and other ports to MS-DOS and Windows, so someone (most probably me) took `DOS' and `WIN' and concatenated them, yielding: #if DOSWIN ... It went for a few years. One day, we were asked by the FSF to avoid this symbol, as it suggested that DOS may win in some way. At the time, this surely meant a lot of places to change, I do not remember if we complied. The fact is that I never had such a pun in head when I made the symbol... -- François Pinard http://www.iro.umontreal.ca/~pinard
participants (6)
-
Brad Clements
-
Fredrik Lundh
-
Greg Ewing
-
Guido van Rossum
-
pinard@iro.umontreal.ca
-
Tim Peters