<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi Liam, I would suggest start out by taking a look at the gRPC quickstart for Python at,
<a href="https://grpc.io/docs/quickstart/python.html">https://grpc.io/docs/quickstart/python.html</a> and then modifying that example to do what you would like.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The Flask server would launch the separate process using multiprocessing. The model process would create a gRPC service endpoint. The Flask server would wait for the model process to start and then establish a gRPC connection as a client
to the gRPC service endpoint of the model process. The gRPC service of the model process would have methods, such as trainModel or getModelStatus, … When an http request occurs on the Flask http server, the server would then invoke the gRPC methods in the
model process.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I hope that helps.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards --Roland<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Liam Geron <liam@chatdesk.com><br>
<b>Date: </b>Thursday, December 20, 2018 at 9:53 AM<br>
<b>To: </b>Roland Hochmuth <rhochmuth@alteryx.com><br>
<b>Cc: </b>Scikit-learn mailing list <scikit-learn@python.org><br>
<b>Subject: </b>Re: [scikit-learn] How to keep a model running in memory?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Hi Roland, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks for the suggestion! I'll certainly look into gRPC or similar frameworks. Currently we have multiprocessing, but it's not used to that same extent. How would the second process have a sort of "listener" to respond to incoming requests
if it is running persistently?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks so much for the help.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Liam<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Dec 20, 2018 at 11:12 AM Roland Hochmuth <<a href="mailto:rhochmuth@alteryx.com">rhochmuth@alteryx.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi Liam, Not sure I have the complete context for what you are trying to do, but have you considered using Python multiprocessing to start a separate process? The lifecycle of that
process could start when the Flask server starts-up or on the first request. The separate process would load and run the model. Depending on what you would like to do, some form of IPC mechanism, such as gRPC could be used to control or get updates from the
model process.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Regards --Roland<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">scikit-learn <scikit-learn-bounces+rhochmuth=<a href="mailto:alteryx.com@python.org" target="_blank">alteryx.com@python.org</a>> on behalf of Aneto <<a href="mailto:aneto@chatdesk.com" target="_blank">aneto@chatdesk.com</a>><br>
<b>Reply-To: </b>Scikit-learn mailing list <<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a>><br>
<b>Date: </b>Thursday, December 20, 2018 at 8:21 AM<br>
<b>To: </b>"<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a>" <<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a>><br>
<b>Cc: </b>Liam Geron <<a href="mailto:liam@chatdesk.com" target="_blank">liam@chatdesk.com</a>><br>
<b>Subject: </b>[scikit-learn] How to keep a model running in memory?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">Hi scikit learn community,</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">We currently use scikit-learn for a model that generates predictions on a server endpoint. We would like to keep the model running in memory instead of
having to re-load the model for every new request that comes in to the server.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">Can you please point us in the right direction for this? Any tutorials or examples.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">In case it's helpful, we use Flask for our web server.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black"><br>
Thank you!</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="color:black">Aneto</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>