[medusa] Re: tweaks to asyn{core,chat}.py

Amos Latteier amos@a...
Fri, 19 Nov 1999 12:09:17 -0800

Robin Becker wrote:
>Sam Rushing wrote:
>>I've been trying to tighten up the async modules a bit to fight an
>>effect that shows up with lots of concurrent connections (>500).
>>I've made some notes:
>> http://www.nightmare.com/medusa/async_tweaks.html
>>I think the changes are a win, so I'll be checking in this code over
>>the next day or so. If anyone out there is violating the
>>asyncore.socket_map abstraction (relying on the way the dictionary is
>>used) or sees a problem I've missed, please let me know.
>>I think the only code in medusa that cares is the stuff in http_server
>>that kills zombie connections. Zope may have done something similar
>>with the ftp server?

A few small changes will need to be made to Zope to accommodate this change
to asycore.socket_map, but not many.

I talked with Jim Fulton about ideas to improve asyncore/asychat select
loop performance a while back. Let's see if I can remember some of his
ideas. (Note, I may have gotten it slightly wrong, sorry Jim ;-)

One problem Zope has with asyncore results from using worker threads in
addition to a medusa thread. There are problems when the worker thread is
ready to go, but the medusa thread is sitting in a select call, which
potentially can take up to 30 secs to return. We get around this by using a
select trigger to wake up select, but we would like to get rid of the need
for a select trigger.

One way to achieve this is to reduce the timeout on the select call to a
small enough value that the worker threads can wait for a timeout without
being overly inconvenienced. This would require the select loop being
adequately optimized.

One optimization Jim suggested is to pass integers to the select loop, not
descriptor objects. I think that this is what you are suggesting as
'fdcache' optimization. Another optimization would be to have a system of
re-entering the select call immediately after returning if nothing is
ready--rather than checking the results and rebuilding the arguments to the
select call. 

>I've been looking at how zombies are handled in Zope. seems that zope
>has an zhttp_channel analogous to http_channel; it has the zombie stuff,
>but without the maintainance thing so the kill_zombies method doesn't
>seem to get called.

I'm pretty sure that it *does* get called. zhttp_channel calls
check_maintenance upon initialization. 

> I noticed that ftp_server doesn't have a zombie
>timeout is there an obvious reason why not?

I've wondered about this too.