[Python-bugs-list] [ python-Bugs-448351 ] coredump in selectmodule.c on Solaris 8

noreply@sourceforge.net noreply@sourceforge.net
Tue, 02 Apr 2002 06:59:09 -0800


Bugs item #448351, was opened at 2001-08-06 04:23
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448351&group_id=5470

Category: Extension Modules
Group: Platform-specific
Status: Open
Resolution: Remind
Priority: 7
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Tim Peters (tim_one)
Summary: coredump in selectmodule.c on Solaris 8

Initial Comment:
I get coredump if I run a small script with Python 2.0
on Solaris 8, compiled with Sun CC Forte 6.1 compiler
(64 bits). I suggest to run it more than once to
produce the error. Purify showed me that there are
reading and writings outside the stack boundary.  

The interesting part of the source:

Modules/selectmodule.c
.
.
static PyObject *
select_select(PyObject *self, PyObject *args)
{
#ifdef MS_WINDOWS
        /* This would be an awful lot of stack space on
Windows! */
        pylist *rfd2obj, *wfd2obj, *efd2obj;
#else
        pylist rfd2obj[FD_SETSIZE + 3];
        pylist wfd2obj[FD_SETSIZE + 3];
        pylist efd2obj[FD_SETSIZE + 3];
#endif
.
.
.
}

In our environment FD_SETSIZE is 65536 as defined in
sys/select.h (see
below). The allocated stack space in select_select is
3*sizeof(rfd2obj)*(FD_SETSIZE+3). It is more than
3Mbytes. The difference between the addresses of the
same variable in two
seperate threads is about 2Mbytes. Lets suppose char
*p1 = (char *)rfd2obj
in thread N and char *p2 = (char *)rfd2obj in thread N
+ 1, abs(p1-p2)
is about 2MB (dbx showed this). The stack is
overwritten between the threads. Is it possible that
the stack size is limited to 2 Mbytes per thread? We
fixed it as solved on Windows allocating these
variables on the heap.

Select.h from Solaris 8.

/usr/include/sys/select.h:
.
.
#ifndef FD_SETSIZE
#ifdef _LP64
#define FD_SETSIZE      65536
#else
#define FD_SETSIZE      1024
#endif  /* _LP64 */
.
.

----------------------------------------------------------------------

>Comment By: Barry Warsaw (bwarsaw)
Date: 2002-04-02 09:59

Message:
Logged In: YES 
user_id=12800

Could this be an off-by-one error in the #if choice for
SELECT_USES_HEAP?  Also, I don't know what the right answer
is for Windows, and I don't have a Solaris 8 box handy (I've
had trouble getting access to the SF compile farm).

I'm reassigning this bug to Tim for pronouncement on troels
Windows question.  If we do drop it to 511, then it'll
probably be fine for Solaris (and I can certainly test it on
Linux).  Otherwise, perhaps we should drop it to 1023.

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2002-04-02 09:44

Message:
Logged In: YES 
user_id=6380

Reopened and raised priority to 7. We should get this sorted
out for 2.2.1 final is released.

----------------------------------------------------------------------

Comment By: Troels Walsted Hansen (troels)
Date: 2002-01-16 13:02

Message:
Logged In: YES 
user_id=32863

Unfortunately 1024 is too much as well. I just finished a 
long session with Purify on Solaris (with 
FD_SETSIZE==1024), and stack corruption occurs unless I 
change the test to "#if FD_SETSIZE > 1023"

Maybe the limit should be lowered to 511 so Windows users 
will get same behaviour as the old code?

I have been testing Python 2.2, browsing CVS tells me the 
code is the same at this point in time.


----------------------------------------------------------------------

Comment By: Barry Warsaw (bwarsaw)
Date: 2001-10-18 15:48

Message:
Logged In: YES 
user_id=12800

Ah I forgot to close this bug report.  This code's been in
the  source long enough to have uncovered problems if there
were any.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-08-16 17:58

Message:
Logged In: YES 
user_id=31435

Back to Barry for re-testing on Linux:  I checked in 
another version that refuses to add 3 to FD_SETSIZE 
anymore.  Fortune favors the bold.

----------------------------------------------------------------------

Comment By: Barry Warsaw (bwarsaw)
Date: 2001-08-16 12:53

Message:
Logged In: YES 
user_id=12800

I've just checked in a minimal patch, taking the easy way
out (i.e. testing for FD_SETSIZE > 1024).  This subtly
changes the behavior on Windows (where Tim says the value is
now 512) since Windows won't normally take the
allocate-on-heap path now.  

selectmodule.c 2.54 has the change.

The changes pass on Linux both for normal, small FD_SETSIZEs
and for artificially cranked FD_SETSIZEs.  Assigning back to
Tim for verification on Windows.

----------------------------------------------------------------------

Comment By: Tim Peters (tim_one)
Date: 2001-08-06 16:52

Message:
Logged In: YES 
user_id=31435

Reassigned to Barry.  I may a good choice to write the 
code, but not to test it (I know zilch about sockets, and 
can't provoke the problem on Windows anyway).

----------------------------------------------------------------------

Comment By: Guido van Rossum (gvanrossum)
Date: 2001-08-06 08:36

Message:
Logged In: YES 
user_id=6380

Dear Anonymous,

a quick workaround is to change the three #ifdef MS_WINDOWS
in selectmodule.c into #if FD_SETSIZE > 1024.

A better idea is currently being discussed on python-dev.


----------------------------------------------------------------------

Comment By: Nobody/Anonymous (nobody)
Date: 2001-08-06 04:31

Message:
Logged In: NO 

The attachment missed, this is the mentioned script:

import threading
import telnetlib

def telnetToHost():
    
        hostname = "my_hostname"
        username = "user_name"
        password = "password"
        
        tn = telnetlib.Telnet(hostname)
        tn.read_until("login: ")
        tn.write(username + '\n')
        tn.read_until("Password: ")
        tn.write(password + '\n')

class MyThread(threading.Thread):
        
        def run(self):
                print "ThreadID", self.cnt, "started"
                telnetToHost()
                print "ThreadID", self.cnt, "finished"

for i in range(0,4):
        m = MyThread()
        m.cnt = i
        m.start()

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=105470&aid=448351&group_id=5470