I'm trying to fix selectmodule.c on Windows (it raises bogus exceptions, because select() on Windows does not set errno). The first patch I had was this: if (n < 0) { + #ifdef MS_WINDOWS + PyErr_SetExcFromWindowsErr(SelectError, WSAGetLastError()); + #else PyErr_SetFromErrno(SelectError); + #endif } else if (n == 0) { /* optimization */ but PyErr_SetExcFromWindowsErr is not present in the 2.2 maintainance branch. An easier fix would be this one, but I wonder if it is allowed/good style to set 'errno': *** 274,279 **** --- 274,282 ---- Py_END_ALLOW_THREADS if (n < 0) { + #ifdef MS_WINDOWS + errno = WSAGetLastError(); + #endif PyErr_SetFromErrno(SelectError); } else if (n == 0) { Thomas
but PyErr_SetExcFromWindowsErr is not present in the 2.2 maintainance branch. An easier fix would be this one, but I wonder if it is allowed/good style to set 'errno':
*** 274,279 **** --- 274,282 ---- Py_END_ALLOW_THREADS
if (n < 0) { + #ifdef MS_WINDOWS + errno = WSAGetLastError(); + #endif PyErr_SetFromErrno(SelectError); } else if (n == 0) {
Well, I'd agree it's not good style - therefore it deserves a comment <wink>. I'd say with a reasonable comment you should just go for it. BDFL-pronouncement-not-withstanding ly, Mark.
Thomas Heller wrote:
I'm trying to fix selectmodule.c on Windows (it raises bogus exceptions, because select() on Windows does not set errno). The first patch I had was this:
if (n < 0) { + #ifdef MS_WINDOWS + PyErr_SetExcFromWindowsErr(SelectError, WSAGetLastError()); + #else PyErr_SetFromErrno(SelectError); + #endif } else if (n == 0) { /* optimization */
but PyErr_SetExcFromWindowsErr is not present in the 2.2 maintainance branch. An easier fix would be this one, but I wonder if it is allowed/good style to set 'errno':
*** 274,279 **** --- 274,282 ---- Py_END_ALLOW_THREADS
if (n < 0) { + #ifdef MS_WINDOWS + errno = WSAGetLastError(); + #endif PyErr_SetFromErrno(SelectError); } else if (n == 0) {
Here's what the man page has to say: NAME errno - number of last error SYNOPSIS #include <errno.h> extern int errno; DESCRIPTION The integer errno is set by system calls (and some library functions) to indicate what went wrong. Its value is significant only when the call returned an error (usually -1), and a library function that does succeed is allowed to change errno. Sometimes, when -1 is also a legal return value one has to zero errno before the call in order to detect possible errors. errno is defined by the ISO C standard to be a modifiable lvalue of type int, and must not be explicitly declared; errno may be a macro. errno is thread-local; setting it in one thread does not affect its value in any other thread. Valid error numbers are all non-zero; errno is never set to zero by any library function. All the error names specified by POSIX.1 must have distinct values. ... Setting errno is allowed; in fact, it is required to set it to 0 sometimes in order to narrow down the location of an error (in a sequence of C library calls). -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/
I'm trying to fix selectmodule.c on Windows (it raises bogus exceptions, because select() on Windows does not set errno).
Are you *sure* about that?
The first patch I had was this: [...] but PyErr_SetExcFromWindowsErr is not present in the 2.2 maintainance branch. An easier fix would be this one, but I wonder if it is allowed/good style to set 'errno':
Yes, assignment to errno is fine. --Guido van Rossum (home page: http://www.python.org/~guido/)
I'm trying to fix selectmodule.c on Windows (it raises bogus exceptions, because select() on Windows does not set errno).
Are you *sure* about that?
Yes. MSDN: The select function returns the total number of socket handles that are ready and contained in the fd_set structures, zero if the time limit expired, or SOCKET_ERROR if an error occurred. If the return value is SOCKET_ERROR, WSAGetLastError can be used to retrieve a specific error code. Thomas
"Thomas Heller" <thomas.heller@ion-tof.com> writes:
Are you *sure* about that? [...}
The select function returns the total number of socket handles that are ready and contained in the fd_set structures, zero if the time limit expired, or SOCKET_ERROR if an error occurred. If the return value is SOCKET_ERROR, WSAGetLastError can be used to retrieve a specific error code.
This is a strong indication, but not enough for certainty. It does not mention errno at all. Regards, Martin
From: "Martin v. Loewis" <martin@v.loewis.de>
"Thomas Heller" <thomas.heller@ion-tof.com> writes:
Are you *sure* about that? [...}
The select function returns the total number of socket handles that are ready and contained in the fd_set structures, zero if the time limit expired, or SOCKET_ERROR if an error occurred. If the return value is SOCKET_ERROR, WSAGetLastError can be used to retrieve a specific error code.
This is a strong indication, but not enough for certainty. It does not mention errno at all.
Yes. Here's an experiment (unpatched python): Python 2.2.1 (#34, Apr 9 2002, 19:34:33) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import select select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (0, 'Error')
Patched python: Python 2.3a0 (#29, Sep 19 2002, 12:38:34) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import select select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10093, 'Either the application has not called WSAStartup, or WSAStartup failed') import socket select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10022, 'An invalid argument was supplied')
Thomas
Patched python:
Python 2.3a0 (#29, Sep 19 2002, 12:38:34) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import select select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10093, 'Either the application has not called WSAStartup, or WSAStartup failed') import socket select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10022, 'An invalid argument was supplied')
Hm... I can confirm this on my Win98SE box. But questions pop up: Why is the error different the first time? And why is this an error at all? On Linux, this is not an error. (In fact, time.sleep() uses this to sleep using subsecond precision.) --Guido van Rossum (home page: http://www.python.org/~guido/)
Patched python:
Python 2.3a0 (#29, Sep 19 2002, 12:38:34) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.
import select select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10093, 'Either the application has not called WSAStartup, or WSAStartup failed') import socket select.select([], [], [], 10) Traceback (most recent call last): File "<stdin>", line 1, in ? select.error: (10022, 'An invalid argument was supplied')
Hm... I can confirm this on my Win98SE box. But questions pop up:
Why is the error different the first time? And why is this an error at all?
The winsock library is not initialized the first time - it seems that socketmodule calls WSAStartup(), but I haven't looked at this in detail. Also I think it's not worth to fix it, there's no use for select() on windows if you don't use sockets - you have to supply at least one socket descriptor (that's the cause for the second error above). Although it could be argued whether it makes sense to simulate a Linux-compatible select for Windows.
On Linux, this is not an error. (In fact, time.sleep() uses this to sleep using subsecond precision.)
Why is the error different the first time? And why is this an error at all?
The winsock library is not initialized the first time - it seems that socketmodule calls WSAStartup(), but I haven't looked at this in detail.
Oh well, that makes some sense.
Also I think it's not worth to fix it, there's no use for select() on windows if you don't use sockets - you have to supply at least one socket descriptor (that's the cause for the second error above).
OK.
Although it could be argued whether it makes sense to simulate a Linux-compatible select for Windows.
Nah, it's been like this for a decade.
On Linux, this is not an error. (In fact, time.sleep() uses this to sleep using subsecond precision.)
From my early Unix (actually Minix) experiments I remember that select(3) was the only possibility to do subsecond delays in Unix. Is this still the same today?
Probably. HAVE_SELECT is the first thing tested in floatsleep(). --Guido van Rossum (home page: http://www.python.org/~guido/)
Also I think it's not worth to fix it, there's no use for select() on windows if you don't use sockets - you have to supply at least one socket descriptor (that's the cause for the second error above).
OK.
Although it could be argued whether it makes sense to simulate a Linux-compatible select for Windows.
Nah, it's been like this for a decade.
In the current form, it breaks asyncore - this is what I wanted to fix in the first place. asyncore contains this code snippet in the poll() function: try: r,w,e = select.select (r,w,e, timeout) except select.error, err: if err[0] != EINTR: raise r = []; w = []; e = [] This will fail on Windows if all of r,w,e are empty. Even if there are active sockets, it may be that this code is executed with all three lists empty. How can this be fixed? I have an SF item at http://www.python.org/sf/611464 discussing this. Thomas
Although it could be argued whether it makes sense to simulate a Linux-compatible select for Windows.
Nah, it's been like this for a decade.
In the current form, it breaks asyncore - this is what I wanted to fix in the first place. asyncore contains this code snippet in the poll() function:
try: r,w,e = select.select (r,w,e, timeout) except select.error, err: if err[0] != EINTR: raise r = []; w = []; e = []
This will fail on Windows if all of r,w,e are empty.
Aargh!!! Apparently asyncore has never worked properly on Windows. Note that it also doesn't check for the Windows error codes on connect().
Even if there are active sockets, it may be that this code is executed with all three lists empty.
Yes.
How can this be fixed?
Change poll() in asyncore.py to use this: if [] == r == w == e: time.sleep(timeout) else: try: r, w, e = select.select(r, w, e, timeout) except select.error, err: ...etc...
I have an SF item at http://www.python.org/sf/611464 discussing this.
The conclusion there seems that select() should be fixed, but then goes on to say that there's no easy way to make it interruptible. Since we don't try to hide the differences between select on Windows and on Unix in other areas (on Windows you can only select on sockets) I'm not sure it's worth trying to fix select if you lose interruptability; fixing asyncore instead is easy enough, and I don't think this is going to bite too many other applications. --Guido van Rossum (home page: http://www.python.org/~guido/)
select on windows is very limited. It is only allowed to be called with socket handles. You cannot use C RTL fd with it or another sort of handle. Because its part of winsock and not part of the C RTL so it cannot mess with errno itself.
HAVE_SELECT is the first thing tested in floatsleep().
HAVE_SELECT should probably be undefined on windows. With the expection that the sockets module for windows can use it. BArry
select on windows is very limited. It is only allowed to be called with socket handles. You cannot use C RTL fd with it or another sort of handle.
Because its part of winsock and not part of the C RTL so it cannot mess with errno itself.
Yes I know.
HAVE_SELECT is the first thing tested in floatsleep().
HAVE_SELECT should probably be undefined on windows. With the expection that the sockets module for windows can use it.
Sorry, floatsleep() on Windows doesn't ever get to testing HAVE_SELECT. So no worry, that part at least works. --Guido van Rossum (home page: http://www.python.org/~guido/)
While we're discussing the non-conformance of Window's select, these 2 errors:
select.error: (10093, 'Either the application has not called WSAStartup, or WSAStartup failed')
select.error: (10022, 'An invalid argument was supplied')
are about the only errors you'll get from select on Windows. Where select would return a socket in the errors list on *nix, on Windows it will come out as readable / writeable, and it's the socket send / rcv that will find out what the problem is. Each version of winsock gets a bit better, but (for example), selecting for write in Win9x-era winsock is essentially a busy-wait. You'll get the socket back immediately, go to write and get the Window's EWOULDBLOCK error. but-heck-it-multitasks-ly-y'rs -- Gordon http://www.mcmillan-inc.com/
From: "Martin v. Loewis" <martin@v.loewis.de>
"Thomas Heller" <thomas.heller@ion-tof.com> writes:
Are you *sure* about that? [...}
The select function returns the total number of socket handles that are ready and contained in the fd_set structures, zero if the time limit expired, or SOCKET_ERROR if an error occurred. If the return value is SOCKET_ERROR, WSAGetLastError can be used to retrieve a specific error code.
This is a strong indication, but not enough for certainty. It does not mention errno at all.
Before we dive into philosophical discussions about what this sentence says, my interpretation would be: If select() returns SOCKET_ERROR, you *should* call WSAGetLastError() to get "details about the problem". Thomas
I'm trying to fix selectmodule.c on Windows (it raises bogus exceptions, because select() on Windows does not set errno).
Are you *sure* about that?
Yes.
MSDN:
The select function returns the total number of socket handles that are ready and contained in the fd_set structures, zero if the time limit expired, or SOCKET_ERROR if an error occurred. If the return value is SOCKET_ERROR, WSAGetLastError can be used to retrieve a specific error code.
Argh! So select() has never returned proper return values on Windows. :-( Thanks for fixing this. Are you gonna fix it in 2.2.2 as well as 2.3? --Guido van Rossum (home page: http://www.python.org/~guido/)
On 24 Sep 2002 at 10:16, Guido van Rossum wrote:
Yes, assignment to errno is fine.
Please see patch 505846. I haven't supplied this patch in proper form yet, but this discussion relates to the patch. I would like to remind folks that on some platforms, one cannot just use "errno = 0". On those platforms calling a function is required to set errno. The point of patch 505846 is to "standardized" the "errno = " function, and secondarily provide a way to "get" the errno. This is done in pyport.h and "all modules" that use or set errno. (not as many as you might think) It's an ugly patch, requires a lot of changes to the core. I'm willing to make all the changes to the core as needed, once we figure out the best way to handle this issue is. In fact, it's this patch that is the principal cause of the "fork python ce" thread also recently discussed in this forum. See "Need advice: cloning python cvs for CE project" Windows CE doesn't allow setting errno. Neither does NetWare (CLIB). Is it worthwhile to discuss patch 505846 some more in this thread? Perhaps those who haven't read the comments on the patch have a clever solution? Or should I just clean up my patch, resubmit it and move on? I agree with Mark's post about keeping CE changes in the core. I'd rather do that. I submitted patch 505846 incorrectly and need to fix it.. But after it's submitted and if accepted, core developers would need to use Py_SetErrno instead of "errno = " And for extension developers. Using the macro would be nice, but it's less of an issue since CE and NetWare ports have to be done "by hand" anyway for these modules, we can make those changes as they're encountered. So .. discuss this, look for better insight, or resubmit the patch and move on? Thanks Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements
Yes, assignment to errno is fine.
Please see patch 505846.
I haven't supplied this patch in proper form yet, but this discussion relates to the patch.
I would like to remind folks that on some platforms, one cannot just use "errno = 0". On those platforms calling a function is required to set errno.
Shucks. That's in violation of the ISO C standard.
The point of patch 505846 is to "standardized" the "errno = " function, and secondarily provide a way to "get" the errno. This is done in pyport.h and "all modules" that use or set errno. (not as many as you might think)
Why also provide an alternative way to get it? Sure you can *get* it even on Win/CE?
It's an ugly patch, requires a lot of changes to the core. I'm willing to make all the changes to the core as needed, once we figure out the best way to handle this issue is.
I have a strong urge to tell you to start porting Linux to your CE hardware rather than bothering with Win/CE. Or buy an iPAQ for which Linux is already available.
In fact, it's this patch that is the principal cause of the "fork python ce" thread also recently discussed in this forum. See "Need advice: cloning python cvs for CE project"
I've given all the advice I have time for.
Windows CE doesn't allow setting errno. Neither does NetWare (CLIB).
Sigh.
Is it worthwhile to discuss patch 505846 some more in this thread? Perhaps those who haven't read the comments on the patch have a clever solution?
Or should I just clean up my patch, resubmit it and move on?
I agree with Mark's post about keeping CE changes in the core. I'd rather do that. I submitted patch 505846 incorrectly and need to fix it.. But after it's submitted and if accepted, core developers would need to use Py_SetErrno instead of "errno = "
Except in extensions that don't have a snowball in hell's chance of working on Win/CE, of course.
And for extension developers. Using the macro would be nice, but it's less of an issue since CE and NetWare ports have to be done "by hand" anyway for these modules, we can make those changes as they're encountered.
So .. discuss this, look for better insight, or resubmit the patch and move on?
As I said, I have a very strong urge to tell you to go away. But I won't. But I really don't like the idea of coding around this particular platform's quirks. --Guido van Rossum (home page: http://www.python.org/~guido/)
Windows CE prevents assignment to errno... There would be a solution if you compiled all the code as C++. (Assuming that C++ reserved words are not used in the python code.) Inject the following definitions: class ErrnoHack { public: operator int(); // return errno value operator =( int ); // assign to errno }; ErrnoHack ErrnoObject #define errno ErrnoObject and you can then write errno = 0; BArry
On 24 Sep 2002 at 20:10, Barry Scott wrote:
Windows CE prevents assignment to errno...
There would be a solution if you compiled all the code as C++. (Assuming that C++ reserved words are not used in the python code.)
Oh that's the other problem I ran into. The core .c has a lot of goto finally; finally: In C mode, this shouldn't matter. But MS's EVT compiler considers "finally" to be a reserved word. I had to change all of these too. (I used local_finally or some such thing) Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements
Oh that's the other problem I ran into.
The core .c has a lot of
goto finally;
finally:
In C mode, this shouldn't matter. But MS's EVT compiler considers "finally" to be a reserved word. I had to change all of these too. (I used local_finally or some such thing)
This compiler seems to fly in the face of the C std whenever it can. What are they trying to accomplish? --Guido van Rossum (home page: http://www.python.org/~guido/)
On 25 Sep 2002 at 11:32, Guido van Rossum wrote:
In C mode, this shouldn't matter. But MS's EVT compiler considers "finally" to be a reserved word. I had to change all of these too. (I used local_finally or some such thing)
This compiler seems to fly in the face of the C std whenever it can. What are they trying to accomplish?
World domination, what else? -- I'm not sure, but I think my latest version of Metrowerks has the same issue. In any case, using "finally" eliminates the possibility of compiling the core as C++, regardless of the compiler being used. I thought I remember seeing a thread on the subject of C++ compilation somewhere. So, the suggested Errno class hack would work, except I still have to change all the finally's. I suspect there are other issues with C++ compilation that I am not aware of. I'm not proposing anything specific here, just rambling. I need to answer your other post. Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements
On Wed, 25 Sep 2002, Brad Clements wrote:
In any case, using "finally" eliminates the possibility of compiling the core as C++, regardless of the compiler being used.
Not in this universe. I think there are already bigger barriers in the way of compiling the Python source as C# or Java... Cheers, M.
On 25 Sep 2002 at 16:41, Michael Hudson wrote:
On Wed, 25 Sep 2002, Brad Clements wrote:
In any case, using "finally" eliminates the possibility of compiling the core as C++, regardless of the compiler being used.
Not in this universe. I think there are already bigger barriers in the way of compiling the Python source as C# or Java...
Uh, how'd we go from C++ to C#/Java ? Brad Clements, bkc@murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements
From: Brad Clements <bkc@murkworks.com>
On 25 Sep 2002 at 16:41, Michael Hudson wrote:
On Wed, 25 Sep 2002, Brad Clements wrote:
In any case, using "finally" eliminates the possibility of compiling the core as C++, regardless of the compiler being used.
Not in this universe. I think there are already bigger barriers in the way of compiling the Python source as C# or Java...
Uh, how'd we go from C++ to C#/Java ?
C++ to C#, I think first passing through managed C++, one of the latest MS inventions <wink>. regards.
Brad> In any case, using "finally" eliminates the possibility of Brad> compiling the core as C++, regardless of the compiler being Brad> used. "finally" is not a keyword in standard C++. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark
Brad Clements <bkc@murkworks.com>:
I'm not sure, but I think my latest version of Metrowerks has the same issue.
Doesn't Metrowerks have a "strict ANSI" switch that turns off all the language extensions? Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+
From: "Greg Ewing" <greg@cosc.canterbury.ac.nz>
Brad Clements <bkc@murkworks.com>:
I'm not sure, but I think my latest version of Metrowerks has the same issue.
Doesn't Metrowerks have a "strict ANSI" switch that turns off all the language extensions?
Yes. ----------------------------------------------------------- David Abrahams * Boost Consulting dave@boost-consulting.com * http://www.boost-consulting.com
Guido> This compiler seems to fly in the face of the C std whenever it can. Guido> What are they trying to accomplish? To extend the language in ways that lock customers into their platform. -- Andrew Koenig, ark@research.att.com, http://www.research.att.com/info/ark
----- Original Message ----- From: "Andrew Koenig" <ark@research.att.com> To: "Guido van Rossum" <guido@python.org> Cc: <bkc@murkworks.com>; <python-dev@python.org> Sent: Wednesday, September 25, 2002 11:38 AM Subject: Re: Reserved keywords in source: was RE: [Python-Dev] Assign to errno allowed?
Guido> This compiler seems to fly in the face of the C std whenever it can. Guido> What are they trying to accomplish?
To extend the language in ways that lock customers into their platform.
The infamous "hijack an open standard by adding proprietary extensions" philosophy, first really publicised by the Halloween documents. regards ----------------------------------------------------------------------- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ Previous .sig file retired to www.homeforoldsigs.com -----------------------------------------------------------------------
participants (14)
-
Andrew Koenig -
Barry Scott -
Brad Clements -
David Abrahams -
Gordon McMillan -
Greg Ewing -
Guido van Rossum -
M.-A. Lemburg -
Mark Hammond -
martin＠v.loewis.de -
Michael Hudson -
Samuele Pedroni -
Steve Holden -
Thomas Heller