<div dir="ltr"><div>Thank you guys for all the info.</div><div><br></div><div>Twisted Web is cool, but there's it's not as interchangeable as WSGI. You can use whatever framework django/flask/webpy/bottle you like and plug it in as long as it's PEP-333 compatible. Not so much for Twisted and Tornado's own ioloop engine.<br>
</div><div><br></div>IPython.parallel <div><br></div><div><a href="http://ipython.org/ipython-doc/stable/install/install.html#dependencies-for-ipython-parallel-parallel-computing" target="_blank">http://ipython.org/ipython-doc/stable/install/install.html#dependencies-for-ipython-parallel-parallel-computing</a><br>
</div><div><br></div><div>It's based on ZeroMQ(PyZMQ), and the `ssh` command. I don't think that's lightweigh enough for busy web clusters.</div><div><br></div><div>By QMachine I assume that's </div>
<div><br></div><div><a href="https://github.com/wilkinson/qmachine" target="_blank">https://github.com/wilkinson/qmachine</a><br></div><div><br></div><div>For web server cluster it's really not a good idea to amplify HTTP requests. One client request amplifies several other HTTP requests on server clusters.</div>
<div><br></div><div>What I propose is something like Zed Shawn's Mongrel2 project (<a href="http://mongrel2.org/">http://mongrel2.org/</a>), use a very lightweight server-side serialization protocol as cluster IPC, you can pass states/data between nodes (workers) easily. It should be agnostic to framework or libraries, the objective is to unite python modules in the realtime web world. Because for request-response web world, a synchronized gateway like WSGI is good enough, between each requests, <a href="https://docs.djangoproject.com/en/dev/faq/general/#does-django-scale">share nothing</a>.</div>
<div><br></div><div>But for realtime web, server side state is very much required. There need to be a fd pool for DBs, external services, and stuff like Server-Side-Push technologies. </div><div><br></div><div>Let's assume the following scenario:</div>
<div><br></div><div>One user submits a blog, his follower gets browser/iOS/Android push notification. Because users are connected different nodes in one big cluster, we need some kind of mechanism to broadcast this message. </div>
<div><br></div><div>In such an architecture we can write simpler code like this:</div><div><br></div><div>from django.db.models.signals import post_save<br></div><div><br></div><div>@receiver(post_save, sender=BlogPostModel)<br>
</div><div>def my_handler(sender, **kwargs):<br></div><div> msg = "User X just posted a new blog, check it out at http://..."</div><div><div> browser_followers.send(msg)</div></div> ios_followers.send(msg)<div>
android_followers.send(msg)<br></div><div><br></div><div>Currently this library reall shines.</div><div><br></div><div><a href="https://pypi.python.org/pypi/telegraphy/">https://pypi.python.org/pypi/telegraphy/</a><br>
</div><div><br></div><div>Telegraphy architecture is like this:</div><div><br></div><div><img src="http://telegraphy.readthedocs.org/en/latest/_images/architecture-protocol-stack.png" alt="Inline image 1"><br></div><div>
<br>
</div><div>What I propose is to merge Web-app part and the AutobahnPython Gateway part into <i>one</i> based on a community honored standard. </div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Mar 17, 2014 at 8:24 PM, anatoly techtonik <span dir="ltr"><<a href="mailto:techtonik@gmail.com" target="_blank">techtonik@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="">On Mon, Mar 17, 2014 at 3:53 AM, est <<a href="mailto:electronixtar@gmail.com">electronixtar@gmail.com</a>> wrote:<br>
> The best we've got is something like Celery, or Telegraphy for Django<br>
<br>
</div>Does IPython suits pattern better?<br>
<a href="http://ipython.org/ipython-doc/stable/parallel/parallel_intro.html" target="_blank">http://ipython.org/ipython-doc/stable/parallel/parallel_intro.html</a><br>
<div class=""><br>
> What do you guys think?<br>
<br>
</div>I think not to reinvent the bicycle. Use PythonJS + QMachine to get to<br>
web client side + distributed processing without abandoning existing<br>
Python skills.<br>
<span class="HOEnZb"><font color="#888888">--<br>
anatoly t.<br>
</font></span></blockquote></div><br></div>