<html style="direction: ltr;">
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<style type="text/css">body p { margin-bottom: 0cm; margin-top: 0pt; } </style>
</head>
<body style="direction: ltr;"
bidimailui-detected-decoding-type="latin-charset" bgcolor="#FFFFFF"
text="#000000">
I think your app is a great use case for Tornado, and a great topic
for a ChiPy talk. ;)<br>
<br>
But I think it's also pretty clearly not a typical "web" (=REST)
application. Which is really the myth I'm trying to dispel: if you
are doing typical "web," async servers are of no particular use to
you, whether for serving files or serving your own generated HTML
entities. You yourself are immune to this particular myth, but I
think the mythical formula "async=scalability" (or, worse,
"async=fast") is pervasive among those developing typical web
applications.<br>
<br>
I also readily agree that threading is not critical for many use
cases: running multiple processes can be just fine, as in your case.
But this assumes that every "worker" is independent and would not
benefit from collaboration, especially via shared memory states. If
you were doing, say, video streaming, there would be a lot of
benefit in having threads coordinate and reuse memory, with a smart
load balancer that knows to send clients watching the same video
stream to the same process. Again, Python's abhorrence of threads is
often quite perfect and simple. Still, it pays to know when it's
just not the best tool for the job.<br>
<br>
<div class="moz-cite-prefix">On 10/11/2012 06:23 PM, Jordan Bettis
wrote:<br>
</div>
<blockquote cite="mid:507754DE.3040200@hafd.org" type="cite">
<pre wrap="">On 10/11/2012 03:00 PM, Tal Liron wrote:
</pre>
<blockquote type="cite">
<pre wrap="">My life mission is to dispel this myth (especially because I used to
believe it myself).
</pre>
</blockquote>
<pre wrap="">
I'm not entirely sure what what myth you're dispelling.
I suppose it's entirely possible that a carefully built synchronous
thread model could be more efficient than an asynchronous epoll worker
process model. I'd think the details would depend on the relative
amounts of bookkeeping and overhead.
Seems irrelevant when discussing Python though, because you'll either be
spinning up and down a ton of interpreter instances or you'll be eaten
by a g(il)rue.
The criticism that tornado is poor at serving files to slow clients also
kinda silly given that nobody does that and I think it's stated pretty
early in the tornado documentation that it's not a good use case.
I was never particularly interested in tornado until my latest project.
I chose it because my old way of doing things fell down, not because
reddit said it was the bee's knees or something.
Anyway, my project has a map tile server builtin to it:
<a class="moz-txt-link-freetext" href="http://urbanfresco.com">http://urbanfresco.com</a>
Originally I just did the tile generation in the worker threads. But the
thing fell apart with only one user if there were a lot of tiles to be
generated, because all the workers would devote themselves to creating
tiles and the site would go unresponsive.
I reworked it so that the mapserver was a tornado requesthandler and the
rest of the site was stuck inside a tornado wsgi handler. I then
separated out the tilemaking to a set of daemons that the mapserver
could send jobs to.
If the tile is in the cache, the mapserver serves it directly, if it's
not, then the map server requests the tilemaker generate it, with a
callback, and then goes back to serving requests until it's done. I have
nginx in front of the whole lot, like so:
client <--> nginx --> static files
|
\-> tornado daemons --> mapserver --> tilemaker daemons
/ \
wsgi app <-/ \-> tile cache
I suppose I could have made a faster system by doing the whole thing as
a multi-threaded C++ app... but given the comparative amount of work
involved I'm very happy with the results of using tornado.
The tornado and tile maker instances are started and stopped using a
script and pid files, just like normal unix daemons. Static files
including uploaded photos are served directly by nginx. I am serving the
tile images through python, so I suppose a slow client could tie up my
tornado processes. I was thinking that nginx would buffer the tiles for
me but I suppose I should make sure of that.
At any rate, my interest in tornado had nothing to do with slow clients.
I have nginx for that. My interest was in separating tile generation out
so it was no longer using up all my workers.
_______________________________________________
Chicago mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Chicago@python.org">Chicago@python.org</a>
<a class="moz-txt-link-freetext" href="http://mail.python.org/mailman/listinfo/chicago">http://mail.python.org/mailman/listinfo/chicago</a>
</pre>
</blockquote>
<br>
</body>
</html>