asyncore deficiencies

[Josiah Carlson]

> I would expect such things to be provided by modules that are
> dependant on asyncore, eg. as asynchat is. It's beyond the scope of
> what asyncore tries to do.

Then what exactly would you like to see in asyncore2?


> I don't need a private map, however given that the other option was a
> module variable, I naturally chose to provide my own to increase any
> required future flexibility. Indeed there is little, if anything to be
> gained from using your own map, but the option was there.


You don't even need to offer a variable to the loop construct.  By 
default, it will use asyncore.map, regardless of where your dispatchers 
are defined.


> The poll function is not documented in the manual, nor do any
> docstrings suggest it is a public interface. Internally asyncore
> chooses which of poll, poll2, or poll3 to use inside loop, so calling
> these directly also bypasses some of asyncore's OS-independance
> functionality.

Seemingly asyncore should do a quick test to see which are available, 
and based on some metrics, automatically select the proper function on 
module import and call it poll_once(), which would be poll, poll2 or 
poll3, based on what the system had available.

It could optionally have a check_again function that should be called 
occasionally, which will, on platforms with more than one select/poll 
available, rebind poll_once based on which would be faster given the 
current size of the socket map.


>>I've never used asyncore with UDP sockets, but maybe using 
>>"handle_connect = handle_read" in your subclass definition could be 
>>sufficient.
> 
> I see this as a workaround, and as such I think the point still
> stands. :)

I suppose it would be a relatively minor patch to check what kind of 
socket it is on initialization, making "handle_connect = handle_read" 
automatic.


>>That is exactly why polling is optional.  Select is available on every 
>>platform, but poll is faster on platforms that support it...
> 
> That is not true at all. If you are feeling investigative, quite a
> complete starting point would be <http://www.kegel.com/c10k.html>.
> There is lots of information there regarding various OS
> implementations async IO performance using poll, epoll, select, aio_*,
> etc.

Well...select is available on linux, windows, solaris (I've used it on 
all three).  I would be surprised if it was not available on all of the 
BSDs, would anyone know for certain?


> That's a good point, but I don't have a problem restarting the poll
> after handling an exception, a) because it shouldn't happen very
> often, b) will probably mean shutting down the service rather than
> trying to continue, and c) exceptions are generally expensive special
> cases anyway.

Socket-exceptions usually mean that a connection has died.  That kind of 
thing can be relatively common.  Depending on your application, setting 
up a callback for your dispatcher instance to notify any controller (if 
necessary) should be relatively easy.


> If an exception is propogating up from the dispatcher, that means the
> dispatcher's method did not handle the exception, which should mean it
> gets passed back up the stack as it does with all other code,
> eventually ending up with a traceback print if it wasn't handled.

Unless of course you want your server/client to stay up under all socket 
circumstances.  Rarely, if ever, do you want your server or client to 
crash when exceptions are raised.  Certainly such behavior can be useful 
when you are initially writing your socket app, but as soon as the thing 
becomes live, likely the last thing you want is for it to crash.


> I dislike the handle_error method since it is essentially redundant in
> the face of Python's own exception handling mechanism. If a dispatcher
> wants it's own function to handle exceptions, it should wrap it's
> methods in a try block.

I believe they had it call handle_error because then you can write 
servers like the following...

#dispatcher subclasses

mydispatcher(listen=('', 9876))
asyncore.loop()

This also allows your various clients and servers to handle exceptions 
differently, and for the exception handling to be implemented in the 
thing that may be raising the exceptions.


> I assume handle_error was provided to allow graceful teardown of a
> dispatcher that was erroneous, but it isn't granular enough to be
> useful for tearing down complicated objects, given an exception that
> could have been generated anywhere in a couple of screenfuls worth of
> code, and an object that could be in any state.

If you do some inspection of the exception (check out the traceback 
module), you can usually find out where an error occurred, and exactly 
what kind of exception it was.


>>>- Decoupling from socket objects: removal of the __getitem__ magic
>>>which causes screenfuls of abuse for a simple typo at the wrong stage
>>>in execution.
>>
>>Infinite recursion in asyncore?  Yikes, I've not hit that bug before. 
>>Care to post some offending code?
> 
> 
> It was entirely my fault, but here is the example anyway. :)

With a bit of 'fancy' stack introspection, that infinite recursion error 
could be easily fixed.  Of course it would end up slowing down attribute 
lookups.

I bet there is a quick fix (without introspection), but I'm too tired to 
figure it out now, and I have jury duty in 9 hours.  Maybe tomorrow evening.


>>One could even have a framework for registering even handlers...yikes, 
>>this is starting to sound like wxPython.  I don't know if that is a good 
>>idea or not.
> 
>     NAME  asyncore
>     Basic infrastructure for asynchronous socket service clients and
> servers.
> 
> I think that answers the question.

Good point, it is a bad idea.


> Is anyone else interested in this? I have already made small
> modifications to the asyncore module to fix a few things, but nothing
> mentioned here. I could put together a proposed asyncore2 module if
> there was interest. I may do it, out of need, anyway. :)

Feel free to post your changes in some sort of public place (on the web 
is probably best).  If people want/need it, they'll probably tell you. 
With enough support, asyncore could get patched or asyncore2 could be born.

  - Josiah