<div dir="ltr"><div>Hello,</div><div><br></div><div>This discussion is pretty interesting to try to list when each architecture is the most efficient, based on the need.</div><div><br></div><div>However, just a small precision: multiprocess/multiworker isn't antinomic with AsyncIO: You can have an event loop in each process to try to combine the "best" of two "worlds".</div><div>As usual in IT, it isn't a silver bullet that will care the cancer, however, at least to my understanding, it should be useful for some business needs like server daemons.</div><div><br></div><div>It isn't a crazy new idea, this design pattern is implemented since a long time ago at least in Nginx: <a href="http://www.aosabook.org/en/nginx.html">http://www.aosabook.org/en/nginx.html</a></div><div><br></div><div>If you are interested in to use this design pattern to build a HTTP server only, you can use easily aiohttp.web+gunicorn: <a href="http://aiohttp.readthedocs.org/en/stable/gunicorn.html">http://aiohttp.readthedocs.org/en/stable/gunicorn.html</a></div><div>If you want to use any AsyncIO server protocol (aiohttp.web, panoramisk, asyncssh, irc3d), you can use API-Hour: <a href="http://www.api-hour.io">http://www.api-hour.io</a></div><div><br></div><div>And if you want to implement by yourself this design pattern, be my guest, if a Python peon like me has implemented API-Hour, everybody on this mailing-list can do that.</div><div><br></div><div>For communication between workers, I use Redis, however, you have plenty of solutions to do that.</div><div>As usual, before to select a communication mechanism you should benchmark based on your use cases: some results should surprise you.</div><div><br></div><div>Have a nice week.</div><div><br></div><div>PS: Thank you everybody for EuroPython, it was amazing ;-)</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div>--<br><div style="font-size:small"><div>Ludovic Gasc (GMLudo)</div></div><div style="font-size:small"><a href="http://www.gmludo.eu/" target="_blank">http://www.gmludo.eu/</a></div></div></div></div></div>
<br><div class="gmail_quote">2015-07-26 23:26 GMT+02:00 Sven R. Kunze <span dir="ltr"><<a href="mailto:srkunze@mail.de" target="_blank">srkunze@mail.de</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    <font face="monospace"><font face="sans-serif">Next update:</font><br>
      <br>
      <br>
      Improving Performance by Running Independent Tasks Concurrently -
      A Survey<br>
      <br>
      <br>
                     | </font><font face="monospace"><font face="monospace">processes</font>               | </font><font face="monospace"><font face="monospace">threads</font>           
              | </font><font face="monospace"><font face="monospace">coroutines</font>  
                 <br>
---------------+-------------------------+----------------------------+-------------------------<br>
    </font><font face="monospace"><font face="monospace"><font face="monospace">purpose        | cpu-bound tasks         |
          cpu- & i/o-bound tasks     | i/o-bound tasks         <br>
        </font>               |                        
        |                            |                         <br>
      </font><span class="">managed by     | os scheduler            | os scheduler +
      interpreter | </span></font><font face="monospace"><font face="monospace">customizable </font>event loop <br>
      controllable   | no                      |
      no                         | yes                     <br>
    </font><font face="monospace"><span class=""><font face="monospace">              
        |                         |                           
        |                         <br>
      </font>parallelism    | yes                     | depends (cf.
      GIL)          | no                      <br></span>
      switching      | at any time             | after any
      bytecode         | at user-defined points  <br><span class="">
      shared state   | no                      |
      yes                        | yes                     <br>
    </span></font><font face="monospace">              
      |                         |                           
      |                         <br>
      startup impact | biggest/medium*         |
      medium                     | smallest                <br>
      cpu impact**   | biggest                 |
      medium                     | smallest                <br><span class="">
      memory impact  | biggest                 |
      medium                     | smallest                <br>
    </span></font><font face="monospace"><font face="monospace"><font face="monospace">               |                        
          |                            |                         <br>
        </font><font face="monospace"><font face="monospace">pool </font>module   
          | multiprocessing.Pool    | multiprocessing.dummy.Pool |
          asyncio.BaseEventLoop   <br>
        </font><font face="monospace"><font face="monospace">solo </font>module   
          | multiprocessing.Process | threading.Thread           |
          ---                     <br>
        </font></font><br>
      <br>
      *<br>
      biggest - if spawn (fork+exec) and always on Windows<br>
      medium - if fork alone<br>
      <br>
      **<br>
      due to context switching</font><span class=""><font face="monospace"><br>
    </font><br>
    <br>
    <div>On 26.07.2015 14:18, Paul Moore wrote:<br>
    </div>
    <blockquote type="cite">
      <pre>Just as a note - even given the various provisos and "it's not that
simple" comments that have been made, I found this table extremely
useful. Like any such high-level summary, I expect to have to take it
with a pinch of salt, but I don't see that as an issue - anyone who
doesn't fully appreciate that there are subtleties, probably wouldn't
read a longer explanation anyway.

So many thanks for taking the time to put this together (and for
continuing to improve it).</pre>
    </blockquote></span>
    You are welcome. :)<span class=""><br>
    <blockquote type="cite">
      <pre>+1 on something like this ending up in the Python docs somewhere.
</pre>
    </blockquote></span>
    Not sure how the process for this is but I think the Python gurus
    will find a way.<br>
  </div>

<br>_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org">Python-ideas@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-ideas" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-ideas</a><br>
Code of Conduct: <a href="http://python.org/psf/codeofconduct/" rel="noreferrer" target="_blank">http://python.org/psf/codeofconduct/</a><br></blockquote></div><br></div>