[pypy-dev] Extension Modules Re: Question on Hard Switching and Soft Switching
andrewfr_ice at yahoo.com
Mon Oct 5 03:22:21 CEST 2009
Hello Richard, Armin et al:
--- On Sun, 10/4/09, Richard Tew <richard.m.tew at gmail.com> wrote:
> In any case, select would not be the right place. It
> is where someone goes to work with asynchronous IO. I would rather
> suggest that the uber-module would be the socket module, and that
> behind the scenes it might use IO completion ports on Windows and
> poll or something similar on other platforms. But the goal in doing
> it, would be to make IO just work with a microthread scheduling
> framework for the user with minimal difficulty or effort.
> In order for this to be so, how it works with the scheduler
>would have to be equally simple for the user.
Yes using IO completion ports or epoll, if done right, is a quick way to
give better performance. However I think the interesting area is analysing how the scheduler interacts with the reactor (assuming we are using a reactor).
I think a strategy will start with looking at stuff like many requests
come in and how fast they can be processed (trying to determine what is a steady state). I will also assume that a good solution, given the aforementioned, is trying minimize stuff like context switches (even though they are fast) and calls to the OS (where one takes a hit). Then there is the issue of how much overhead is incurred from laying, that is locating "networking smarts" in a tasklet, as opposed to the interpreter.
Again, as I stated in a previous post, I don't have a good conceptual framework. I have read some papers on high performance server design. Would have to read more. I also suspect a lot of experimentation would have to be done to see what works.
More information about the Pypy-dev