<HTML><HEAD></HEAD>
<BODY dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 10pt; FONT-FAMILY: 'Lucida Sans'; COLOR: #000000">
<DIV>Thanks for your reply, Raphael.</DIV>
<DIV> </DIV>
<DIV>Here’s some code using the Boston dataset to reproduce this. </DIV>
<DIV> </DIV>
<DIV>=== START CODE ===</DIV>
<DIV>import numpy as np</DIV>
<DIV>from sklearn import datasets</DIV>
<DIV>from sklearn.ensemble import GradientBoostingRegressor</DIV>
<DIV> </DIV>
<DIV>boston = datasets.load_boston()</DIV>
<DIV>X, Y = (boston.data, boston.target)</DIV>
<DIV> </DIV>
<DIV>n_estimators = 712 </DIV>
<DIV># Note: From 712 onwards, the feature importance sum is less than 1</DIV>
<DIV> </DIV>
<DIV>params = {'n_estimators': n_estimators, 'max_depth': 6, 'learning_rate':
0.1}</DIV>
<DIV>clf = GradientBoostingRegressor(**params)</DIV>
<DIV>clf.fit(X, Y)</DIV>
<DIV> </DIV>
<DIV>feature_importance_sum = np.sum(clf.feature_importances_)</DIV>
<DIV>print "At n_estimators = %i, feature importance sum = %f" % (n_estimators ,
feature_importance_sum)</DIV>
<DIV> </DIV>
<DIV>=== END CODE ===</DIV>
<DIV> </DIV>
<DIV>If we deem this to be an error, I can open a bug to track it. Please
share your thoughts on it.</DIV>
<DIV> </DIV>
<DIV>Thank you,</DIV>
<DIV>-Doug</DIV>
<DIV> </DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>
<DIV style="FONT: 10pt tahoma">
<DIV> </DIV>
<DIV style="BACKGROUND: #f5f5f5">
<DIV style="font-color: black"><B>From:</B> <A title=drraph@gmail.com
href="mailto:drraph@gmail.com">Raphael C</A> </DIV>
<DIV><B>Sent:</B> Tuesday, August 30, 2016 11:28 PM</DIV>
<DIV><B>To:</B> <A title=scikit-learn@python.org
href="mailto:scikit-learn@python.org">Scikit-learn user and developer mailing
list</A> </DIV>
<DIV><B>Subject:</B> Re: [scikit-learn] Gradient Boosting: Feature Importances
do not sum to 1</DIV></DIV></DIV>
<DIV> </DIV></DIV>
<DIV
style='FONT-SIZE: small; TEXT-DECORATION: none; FONT-FAMILY: "Calibri"; FONT-WEIGHT: normal; COLOR: #000000; FONT-STYLE: normal; DISPLAY: inline'>Can
you provide a reproducible example?
<DIV>Raphael<BR><BR>On Wednesday, August 31, 2016, Douglas Chan <<A
href="mailto:douglas.chan@ieee.org">douglas.chan@ieee.org</A>> wrote:<BR>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<DIV dir=ltr>
<DIV dir=ltr>
<DIV style="FONT-SIZE: 10pt; FONT-FAMILY: 'Lucida Sans'; COLOR: #000000">
<DIV>Hello everyone,</DIV>
<DIV>
<DIV> </DIV>
<DIV>I notice conditions when Feature Importance values do not add up to 1 in
ensemble tree methods, like Gradient Boosting Trees or AdaBoost Trees. I
wonder if there’s a bug in the code.</DIV>
<DIV> </DIV>
<DIV>This error occurs when the ensemble has a large number of
estimators. The exact conditions depend variously. For example,
the error shows up sooner with a smaller amount of training samples. Or,
if the depth of the tree is large. </DIV>
<DIV> </DIV>
<DIV>When this error appears, the predicted value seems to have
converged. But it’s unclear if the error is causing the predicted value
not to change with more estimators. In fact, the feature importance sum
goes lower and lower with more estimators thereafter.</DIV>
<DIV> </DIV>
<DIV>I wonder if we’re hitting some floating point calculation error. </DIV>
<DIV> </DIV>
<DIV>Looking forward to hear your thoughts on this.</DIV>
<DIV> </DIV>
<DIV>Thank you!</DIV>
<DIV>-Doug</DIV>
<DIV> </DIV></DIV></DIV></DIV></DIV></BLOCKQUOTE></DIV>
<P>
<HR>
_______________________________________________<BR>scikit-learn mailing
list<BR>scikit-learn@python.org<BR>https://mail.python.org/mailman/listinfo/scikit-learn<BR></DIV></DIV></DIV></BODY></HTML>