[Spambayes] sb_tray eating time and space with unhandled exceptions

Tony Meyer tameyer at ihug.co.nz
Wed Mar 9 02:38:02 CET 2005


> Sorry, Tony, this must be a pain for you.

Not your fault.  This is a result of changing to Python 2.4 (2.3's asynchat
has no calls to handle_expt(), so it's not possible to have this happen).
So it's my fault for doing so - I should have stuck with 2.3 for 1.0.x and
only moved to 2.4 for 1.1.x (the 1.0.2 fiasco would have been avoided, too).
If only the email package hadn't improved so much from 2.3 to 2.4...

> I'm quite happy to wait until you have time to address the issue 
> yourself - forward progress is generally better than patching up, and 
> you have suggested useful workarounds.
[...]

I wouldn't want to be responsible for any trouble with Pycon <wink>, so I've
looked into it now.  Comparing 2.3 and 2.3's asyncore, 2.4 passes all the
read/write sockets to select.select(), which 2.3 didn't, and this is where
the problem is.  MSDN tells me that select() will indicate a problem when a
connect call fails.

<http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/wi
nsock/select_2.asp>

And that was enough for me to figure out how to duplicate it.  If I
configure my SpamBayes to connect to my POP3 server by IP address rather
than domain name, then I get this error when I don't have network access.
(If I use the domain name, then the lookup fails and I get a "getaddrinfo
failed" error, which is nicely handled, instead.)

Looking deeper: because asyncore.dispatcher creates non-blocking sockets,
then the connect() call's call to socket.connect_ex() will normally
(according to MSDN, again) return EWOULDBLOCK, which asyncore.dispather
considers good enough to keep going with.  However, that means that if the
connect actually fails, then it's not until select.select() is called that
the error occurs.  If that happens, then the error is logged, and
asyncore.loop() tries to poll again, and the error occurs again.  loop()
loops fast enough to get about 10,000 attempts per second, it seems :/

<http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winsock/wi
nsock/windows_sockets_error_codes_2.asp>

For SpamBayes, the solution is to set the socket to blocking while doing the
connect, so that errors arise if necessary, and then set it to nonblocking
after a successful connect.  This fix seems to work for me, and I'll check
it in.

So then, there are two questions: (1) given that this problem will occur any
time someone tries to make a connection when it will fail, is a 1.0.4
release called for, and (2) is this really what asyncore wants to be doing.

I think I'll ask (2) on python-list and see what the response is.

=Tony.Meyer

-- 
Please always include the list (spambayes at python.org) in your replies
(reply-all), and please don't send me personal mail about SpamBayes.
http://www.massey.ac.nz/~tameyer/writing/reply_all.html explains this.



More information about the Spambayes mailing list