[Chicago] Prudence

[Tal Liron]

  The truth is that non-blocking I/O is not a big deal for web 
applications. Somewhere at the top, you /always/ have a thread pool. In 
order to do anything meaningful to generate a page dynamically, you will 
have a single thread going through some code, which you'll want to 
release ASAP. A user generates a hit with a browser, and generates a 
tiny response. Breaking it up to multiple calling threads will not help 
anything -- it will just create a data synchronization burden for you, 
which is already considerable in a multi-threaded environment, such as 
Prudence.

When it comes down to it, there are actually very few use cases for pure 
non-threaded I/O on the Internet: streaming media, or media in general 
-- basically anything /static/. Prudence does have (experimental) 
support in version 1.0 for something called "deferred conversations," in 
which you can release the current thread and commit it whenever you feel 
like it. But ... what kind of user experience is that? The user will be 
waiting for your web page to return, and meanwhile watch the browser 
spin. If you have a potentially long request, you are far better off 
putting something on a task queue, and returning a "please wait" page to 
the user, which will continue polling your site until it gets a result. 
Think of something like Travelocity's "searching for you flights" page.

I deal with the issue in some depth in the chapter "Scaling Tips" in the 
Prudence Manual:

http://threecrickets.com/prudence/scaling/#toc-Subsection-89

Where non-blocking I/O can help a pure web app is: delivering media (if 
you really want to do it from the same server... not necessarily 
recommended, but possible), and more graceful handling of failure. In my 
ideal, Prudence's I/O frontend would be able to talk directly to the 
caching layer (on the roadmap), but even that is hard to recommend as 
such a great feature. If the thread pool handling the "dynamic" web 
pages is saturated, and you're just returning static cached pages to 
some users, then your application may seem to work, but is in fact quite 
broken. You haven't solved your scaling problem, just put a bandaid on it.

Also, though non-blocking I/O is interesting (and hard to do) it's not 
even that necessary anymore. Modern kernels (Linux especially) do a 
fabulous job handling many thousands of rotating threads. If you look at 
a few of the high-scalability web servers available for Linux, some of 
the best performing and scaling ones are those based on threads, not async.

-Tal

On 09/27/2010 10:21 AM, Daniel Griffin wrote:
> +1 for something really cool.
>
> How do you reconcile non-blocking I/O with blocking database calls?
>
>
> On Mon, Sep 27, 2010 at 10:17 AM, Trent Jurewicz <tjurewicz at gmail.com 
> <mailto:tjurewicz at gmail.com>> wrote:
>
>     +1 to hear more about prudence in general and the caching layer
>     implemented...
>
>
>     On Mon, Sep 27, 2010 at 3:18 AM, Tal Liron
>     <tal.liron at threecrickets.com <mailto:tal.liron at threecrickets.com>>
>     wrote:
>
>
>
>         Hey ChiPy,
>
>
>         After more than a year of development, I'm happy to announce a
>         release candidate for Prudence:
>
>
>         http://threecrickets.com/prudence/
>
>
>         Yes, yet another web development framework for Python. Yawn.
>         But this one is a bit different:
>
>
>         1. Embraces the JVM. That means Jython-centric, with some
>         limited support for Jepp (real CPython on the JVM, bleh).
>
>         2. Designed ground-up for REST. Nothing about HTTP is hidden
>         from you. You know, HTTP is actually really useful.
>
>         3. Non-blocking I/O and other adult thinking about concurrency.
>
>         4. Probably the most sophisticated caching system you've seen
>         in a web development platform.
>
>         5. No data layer -- add your own. The example app uses
>         SQLAlchemy. (I have other apps talking to MongoDB.)
>
>         6. Ready-to-rumble package -- just download, and will run on
>         any JVM platform. Tested on Linux, OpenSolaris, OS X and
>         Windows. Ubuntu repository FTW.
>
>
>         I'd be happy to give a presentation to ChiPy on "Advanced
>         Caching with Prudence". Even if you never use Prudence, you
>         might get jealous and try to implement some of these tricks in
>         your own platform of choice.
>
>
>         Can I get a +1?
>
>
>         A future presentation could also be about Scripturian, the
>         underlying library that makes Jython work well in highly
>         concurrent environments. I had to do a lot of hacking in the
>         Jython source code, and learn things that no man should have
>         to know.
>
>
>         Love,
>
>         Tal
>
>         _______________________________________________
>         Chicago mailing list
>         Chicago at python.org <mailto:Chicago at python.org>
>         http://mail.python.org/mailman/listinfo/chicago
>
>
>
>     _______________________________________________
>     Chicago mailing list
>     Chicago at python.org <mailto:Chicago at python.org>
>     http://mail.python.org/mailman/listinfo/chicago
>
>
>
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago