Threading server exits ERESTARTNOHAND

[Rune Hansen]

Rune Hansen wrote:
> select(0, NULL, NULL, NULL, {0, 8000})  = 0 (Timeout)
> futex(0x822f530, FUTEX_WAKE, 1)         = 0
> gettimeofday({1075823054, 603000}, NULL) = 0
> futex(0x822f530, FUTEX_WAKE, 1)         = 0
> select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
> futex(0x822f530, FUTEX_WAKE, 1)         = 0
> gettimeofday({1075823054, 623210}, NULL) = 0
> futex(0x822f530, FUTEX_WAKE, 1)         = 0
> select(0, NULL, NULL, NULL, {0, 30370}) = ? ERESTARTNOHAND (To be 
> restarted)
> +++ killed by SIGSEGV +++
> 
> This is the output from strace -o xx python2.3 Spider.py, Spider.py 
> being a threaded web crawler. The server segfaults at no specific 
> interval, running for 30 secs and up to 10-15 minutes.
> 
> I can't figure out what's going on. In my years of threaded programming 
> this is the first time I've seen it (if it makes me a novice, please 
> don't rub it in :) I've found some information about "ERESTARTNOHAND" 
> (one hint was /etc/security/limits.conf) and gotten the feeling that it 
> has more to do with RedHat 9.0 than python2.3.3, but I'd really like to 
> hear from someone who perhaps has got some experience with this.
> 
> Server and setup:
> Compiled Python2.3.3 on RedHat 9.0 with default Athlon kernel.
> 
> The server starts a XMLRPCServer and Controller. The Controller starts 
> 15 threads doing urllib2.urlopen with data, using queue.get_nowait(), 
> received on the XMLRPCServer . Each thread has it's own MySQLdb 
> connection doing INSER DELAYED.
> mxTidy is used before data is fed to an HTMLParser instance.
> 
> regards
> 
> /rune
> 

In answer to my own question I'd like to quote our benevolent dictator:

"In any project that is multi-threaded, most bugs will come from 
threading issues.
This is regardless of programming language -- it's a deep, as yet 
ununderstood property of threads."
-- Guido van Rossum

So, there you are. Lesson learned.

regards

/rune