<div dir="ltr">Something like<div><br></div><div> your_df['prediction'] = pd.Series(clf.predict(X_test), index=X_test.index)</div><div><br></div><div>should handle all the alignment.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 20, 2017 at 11:04 AM, Ruchika Nayyar <span dir="ltr"><<a href="mailto:ruchika.work@gmail.com" target="_blank">ruchika.work@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The original dataset contains both trainng/testing, I have predictions only on testing dataset. If I do what you suggest <div>will it preserve indexing?</div></div><div class="gmail_extra"><br clear="all"><div><div class="m_5705832060634146426gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><font face="tahoma,sans-serif">Thanks,<br>Ruchika<br></font><font face="tahoma,sans-serif"><br></font></div></div></div></div></div></div><div><div class="h5">
<br><div class="gmail_quote">On Thu, Jul 20, 2017 at 11:37 AM, Julio Antonio Soto de Vicente <span dir="ltr"><<a href="mailto:julio@esbet.es" target="_blank">julio@esbet.es</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div>Hi Ruchika,</div><div id="m_5705832060634146426m_1483527591689507962AppleMailSignature"><br></div><div id="m_5705832060634146426m_1483527591689507962AppleMailSignature">The predictions outputted by all sklearn models are just 1-d Numpy arrays, so it should be trivial to add it to any existing DataFrame:</div><div id="m_5705832060634146426m_1483527591689507962AppleMailSignature"><br></div><div id="m_5705832060634146426m_1483527591689507962AppleMailSignature">your_df["prediction"] = clf.predict(X_test)<br><br>--<div>Julio</div></div><div><div class="m_5705832060634146426h5"><div><br>El 20 jul 2017, a las 17:23, Ruchika Nayyar <<a href="mailto:ruchika.work@gmail.com" target="_blank">ruchika.work@gmail.com</a>> escribió:<br><br></div><blockquote type="cite"><div><div dir="ltr">Hi Scikit-learn Users, <div><br></div><div>I am analyzing some proxy logs to use Machine learning to classify the events recorded as either "OBSERVED" or "BLOCKED". This is a little snippet of my code: </div><div>The input file is a csv with tokenized string fields. </div><div><br></div><div>**************</div><div># load the file </div><div><div>M = pd.read_csv("output100k.csv").<wbr>fillna('')</div></div><div><br></div><div># define the fields to use </div><div><div>min_df = 0.001</div><div>max_df = .7</div><div>TxtCols = ['request__tokens', 'requestClientApplication__tok<wbr>ens',</div><div> 'destinationZoneURI__tokens',<wbr>'cs-categories__tokens', </div><div> 'fileType__tokens', 'requestMethod__tokens','tcp_s<wbr>tatus1',</div><div> 'app','tcp_status2','dhost'</div><div> ]</div></div><div>NumCols = ['rt', 'out', 'in', 'time-taken','rt_length', 'dt_length']<br></div><div><br></div><div># vectorize the fields </div><div>TfidfModels = [TfidfVectorizer(min_df = min_df, max_df=max_df).fit(M[t]) for t in TxtCols]<br></div><div><br></div><div><div># define the columns of sparse matrix </div><div>X = hstack([m.transform(M[n].filln<wbr>a('')) for m,n in zip(TfidfModels, TxtCols)] + \</div><div> [csr_matrix(pd.to_numeric(M[n<wbr>]).fillna(-1).values).T for n in NumCols])</div><div> </div><div># target variable </div><div>Y = M.act.values </div></div><div><br></div><div>## Define train/test parts and scale them <br></div><div><div>X_train, X_test, y_train, y_test = tts(X, Y, test_size=0.2)</div><div>scaler = StandardScaler(with_mean=False<wbr>, with_std=True)</div><div>scaler.fit(X_train)</div><div>X_train=scaler.transform(X_tra<wbr>in)</div><div>X_test=scaler.transform(X_test<wbr>)</div></div><div><br></div><div><div><div class="m_5705832060634146426m_1483527591689507962gmail_signature"><div dir="ltr"><div dir="ltr"><div><font face="tahoma,sans-serif"><br></font></div><div><font face="tahoma,sans-serif"># define the model and train </font></div><div><font face="tahoma,sans-serif">clf = MLPClassifier(activation='logi<wbr>stic', solver='lbfgs').fit(X_train,y_<wbr>train)<br></font></div><div><font face="tahoma,sans-serif"><div># use the model to predict on X_test and convert into a data frame </div><div>df=pd.DataFrame(clf.predict(X_<wbr>test))<br></div><div><br></div><div>**</div><div><pre>199845 OBSERVED
199846 OBSERVED</pre><pre>[199847 rows x 1 columns]></pre><pre>**</pre></div></font></div><div><font face="tahoma,sans-serif">Now at the end I have a DataFrame with 20K entries with just one column </font></div><div><font face="tahoma,sans-serif">"Label", how di I connect it to the main dataframe M, since I want to do some </font></div><div><font face="tahoma,sans-serif">investigations on this outcome ?</font></div><div><font face="tahoma,sans-serif"><br></font></div><div><font face="tahoma,sans-serif">Any help? </font></div><div><font face="tahoma,sans-serif"><br></font></div><div><font face="tahoma,sans-serif">Thanks,<br>Ruchika<br></font><font face="tahoma,sans-serif"><br></font></div></div></div></div></div>
</div></div>
</div></blockquote></div></div><blockquote type="cite"><div><span>______________________________<wbr>_________________</span><br><span>scikit-learn mailing list</span><br><span><a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a></span><br><span><a href="https://mail.python.org/mailman/listinfo/scikit-learn" target="_blank">https://mail.python.org/mailma<wbr>n/listinfo/scikit-learn</a></span><br></div></blockquote></div><br>______________________________<wbr>_________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailma<wbr>n/listinfo/scikit-learn</a><br>
<br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
<br></blockquote></div><br></div>