<div dir="ltr">Hi Thomas,<div><br></div><div>A number of people I've learned from have given me the following "recipe", which I hold to loosely.</div><div><ol><li>Start with Random Forest - it should be able to give you good baseline predictive capacity. </li><li>Let's say you don't care about interpretability, but only care about predictive value. Keep tweaking RF parameters (use grid search + cross validation), or switch to gradient boosting.</li><li>Let's say you do care about interpretability. Use RF's feature_importances_ to get out the features that are important for prediction. Try linear regression on just those, may also want to try multiplying those features together to get the "interaction" product of those features. (this is using RF as a feature selection method).</li></ol><div>Beyond this, I am sure more "expert" types will be able to chime in, and also correct me if I've said anything wrong here.</div></div><div><br></div><div>Cheers</div><div>Eric</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 1, 2016 at 10:59 AM, Thomas Evangelidis <span dir="ltr"><<a href="mailto:tevang3@gmail.com" target="_blank">tevang3@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)">Dear <span id="m_3993021503299341370:2go.1">scikit</span>-learn users and developers,</div><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)">I have a <span id="m_3993021503299341370:2go.2">dataset</span> consisting of 42 observation (<span id="m_3993021503299341370:2go.3">molnames</span>) and 4 variables (<span id="m_3993021503299341370:2go.4">VDWAALS</span>, EEL, <span id="m_3993021503299341370:2go.5">EGB</span>, <span id="m_3993021503299341370:2go.6">ESURF</span>) with which I want to make a predictive model that estimates the experimental value (<span id="m_3993021503299341370:2go.7">Expr</span>). I tried multivariate linear regression using 10,000 bootstrap repeats each time using 21 observations for training and the rest 21 for testing, but the average correlation was only R= 0.1727 +- 0.19779.</div><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-size:small;color:rgb(0,0,0)"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">molname VDWAALS EEL EGB ESURF Expr<br>CHEMBL108457 -20.4848 -96.5826 23.4584 -5.4045 -7.27193<br>CHEMBL388269 -50.3860 28.9403 -51.5147 -6.4061 -6.8022<br>CHEMBL244078 -49.1466 -21.9869 17.7999 -6.4588 -6.61742<br>CHEMBL244077 -53.4365 -32.8943 34.8723 -7.0384 -6.61742<br>CHEMBL396772 -51.4111 -34.4904 36.0326 -6.5443 -5.82207<br>........</blockquote><div><br></div><div>I would like your advice about what other machine learning algorithm I could try with these data. E.g. can I make a decision tree or the observations and variable are too few to avoid overfitting? I could include more variables but the observations will always remain 42.</div><div><br></div><div>I would greatly appreciate any advice!</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Thomas</div></font></span></div><div class="m_3993021503299341370gmail_signature"><div dir="ltr"><div><div dir="ltr"><p style="margin-bottom:0cm" align="LEFT">
</p>
</div></div></div></div>
</div>
<br>______________________________<wbr>_________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
<br></blockquote></div><br></div>