[Python-Dev] Another 2.4 asyncore headache

[Tim Peters]

Another change to asyncore broke Zope:

    http://collector.zope.org/Zope3-dev/261

I may have mentioned before that asyncore gives me a headache.

The checkin comment for rev 1.57 just says:

    In poll(), check connections for exceptional conditions

and it's not mentioned in NEWS or in any doc changes.  I don't want to
guess what the intent was.  It definitely has a bad effect on Zope.

I can't guess what asyncore's intent is in the presence of threads
either.  It appears to be robust against other threads mutating the
socket map *across* select() calls.  But Zope appears to mutate the
socket map between the time map.items() captures a snapshot and the
time select() is called.  There's a thread race here when an object in
the socket map is closing, and removing itself from the socket map. 
In 2.3.4 it apparently didn't matter because the exception set passed
to select() was always empty.  In 2.4 it's passing a non-readable
non-writable object in select's exception set, and Zope manages to
close the underlying socket before select() is called.  The result is
a select() error that (a) couldn't happen in previous Pythons, and (b)
Zope sure isn't expecting.

More generally, in 2.3.4 it didn't hurt to leave all kinds of crap in
the socket map, provided that any such crap didn't call itself ready
to read or ready to write.  select() would never see it then.  In 2.4,
select() always sees it.  That's a big change, particularly because
everyone I've seen wrestling with asyncore works via iterative
poke-and-hope, fiddling their code more-or-less randomly, until
mysterious errors stop appearing.

So if it's of real value to you (I don't know the motivating use case
for the change) to pass stuff in the exception set, would it be good
enough to do

    e = r + w

instead of (in effect)

    e = map.keys()

?  Then stuff that doesn't call itself readable or writable would
again, as in 2.3.4, not get passed to select().

Or Zope has to change.