select() call and filedescriptor out of range in select error

[Steven D'Aprano]

On Wed, 15 Sep 2010 21:05:49 -0700, k3xji wrote:

> Hi all,
> 
> We have a select-based server written in Python. Occasionally, maybe
> twice a month there occurs a weird problem, select() returns with
> filedescriptor out of range in select() error. This is of course a
> normal error and handled gracefully. Our policy is to take down few
> users for select() to handle the next cycle. However, once this error
> occurs, this also fails too:
> 
> self.__Sockets.remove(socket)
> 
> self.__Socket's is the very basic list of sockets we use in our IO loop.
> The call fails with:
> remove(x): x not in list


Please show the *exact* error message, including the traceback, by 
copying and pasting it. Do not retype it by hand, or summarize it, or put 
it into your own words.


 
> First of all, in our entire application there is no line of code like
> remove(x), meaning there is no x variable. 

Look at this example:

>>> sockets = []
>>> sockets.remove("Hello world")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: list.remove(x): x not in list


"x" is just a placeholder. It doesn't refer to an actual variable x.


> Second, the Exception shows
> the line number containing above code. So self.__Sockets.remove(socket)
> this fails with remove(x): x not in list....

Exactly.


 
> I cannot understand the problem. It happens in sporadic manner and it
> feels that the ValueError of select() call somehow corrupts the List
> structure itself in Python? Not sure if something like that is possible.

Anything is possible, but it's not likely. What's far more likely is that 
you have a bug in your code, and that somehow, under rare circumstances, 
it tries to remove something from a list that was never inserted into the 
list. Or it tries to remove it twice.

My guess is something like this:

try:
    socket = get_socket()
    self._sockets.append(socket)
except SomeError:
    pass
# later on
self._sockets.remove(socket)




-- 
Steven