<div dir="auto"><div dir="ltr">That's an excellent discussion! I've always wondered how other tools like R handled naturally categorical variables or not. LightGBM has a scikit-learn API which handles categorical features by inputting their columns names (or indexes):<div>```</div><div>import lightgbm</div><div dir="auto">lgb=lightgbm.LGBMClassifier()</div><div dir="auto">lgb.<span style="max-width:100%;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;border-width:initial;border-color:initial">fit</span>(<font color="#555555" face="lato, proxima-nova, helvetica neue, arial, sans-serif"><span style="font-size:14.4px"><b>X</b></span></font>, <font color="#555555" face="lato, proxima-nova, helvetica neue, arial, sans-serif"><span style="font-size:14.4px"><b>y</b></span></font>, <font color="#555555" face="lato, proxima-nova, helvetica neue, arial, sans-serif"><span style="font-size:14.4px"><b>feature_name=... </b></span></font>, <font color="#555555" face="lato, proxima-nova, helvetica neue, arial, sans-serif"><span style="font-size:14.4px"><b>categorical_feature=... </b></span></font>)</div><div dir="auto"><br></div><div dir="auto">``` </div><div dir="auto"><br></div><div dir="auto">Where:</div><div dir="auto"><br></div><div dir="auto">- feature_name (list of strings or 'auto', optional (default='auto')) – Feature names. If ‘auto’ and data is pandas DataFrame, data columns names are used.</div><div dir="auto"><br></div><div dir="auto">- categorical_feature (list of strings or int, or 'auto', optional (default='auto')) – Categorical features. If list of int, interpreted as indices. If list of strings, interpreted as feature names (need to specify feature_name as well). If ‘auto’ and data is pandas DataFrame, pandas unordered categorical columns are used. All values in categorical features should be less than int32 max value (2147483647). </div><div><br></div><div><br></div><div>As a suggestion, Scikit-Learn could add a `categorical_feature` parameter in the tree-based estimators in order to work on the same way. </div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, May 1, 2020 at 12:54 PM C W <<a href="mailto:tmrsg11@gmail.com" target="_blank" rel="noreferrer">tmrsg11@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thank you for the link, Guilaumme. In my particular case, I am working on random forest classification.<div><br></div><div>The notebook seems great. I will have to go through it in detail. I'm still fairly new at using sklearn.</div><div><br></div><div>Thank you for everyone's quick response, always feeling loved on here! :)<br><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, May 1, 2020 at 4:00 AM Guillaume Lemaître <<a href="mailto:g.lemaitre58@gmail.com" target="_blank" rel="noreferrer">g.lemaitre58@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">OrdinalEncoder is the equivalent of pd.factorize and will work in the scikit-learn ecosystem.<div><br></div><div>However, be aware that you should not just swap OneHotEncoder to OrdinalEncoder just at your wish.</div><div>It depends of your machine learning pipeline.</div><div><br></div><div>As mentioned by Gael, tree-based algorithm will be fine with OrdinalEncoder. If you have a linear model,</div><div>then you need to use the OneHotEncoder if the categories do not have any order.</div><div><br></div><div>I will just refer to one notebook that we taught in EuroScipy last year:</div><div><a href="https://github.com/lesteve/euroscipy-2019-scikit-learn-tutorial/blob/master/rendered_notebooks/02_basic_preprocessing.ipynb" target="_blank" rel="noreferrer">https://github.com/lesteve/euroscipy-2019-scikit-learn-tutorial/blob/master/rendered_notebooks/02_basic_preprocessing.ipynb</a><br></div><div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 1 May 2020 at 05:11, C W <<a href="mailto:tmrsg11@gmail.com" target="_blank" rel="noreferrer">tmrsg11@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hermes,<div><br></div><div>That's an interesting function. Does it work with sklearn after factorize? Is there any example? Thanks!</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 30, 2020 at 6:51 PM Hermes Morales <<a href="mailto:paisanohermes@hotmail.com" target="_blank" rel="noreferrer">paisanohermes@hotmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Perhaps pd.factorize could hello?<br>
<br>
</div>
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
<span id="m_-7152252324534018961gmail-m_7674425925647437139gmail-m_-3048802919839539925gmail-m_136390889793816078gmail-m_555890274124597800OutlookSignature">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Obtener <a href="https://aka.ms/ghei36" target="_blank" rel="noreferrer">Outlook para Android</a></div>
</span><br>
</div>
<hr style="display:inline-block;width:98%">
<div id="m_-7152252324534018961gmail-m_7674425925647437139gmail-m_-3048802919839539925gmail-m_136390889793816078gmail-m_555890274124597800divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> scikit-learn <scikit-learn-bounces+paisanohermes=<a href="mailto:hotmail.com@python.org" target="_blank" rel="noreferrer">hotmail.com@python.org</a>> on behalf of Gael Varoquaux <<a href="mailto:gael.varoquaux@normalesup.org" target="_blank" rel="noreferrer">gael.varoquaux@normalesup.org</a>><br>
<b>Sent:</b> Thursday, April 30, 2020 5:12:06 PM<br>
<b>To:</b> Scikit-learn mailing list <<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a>><br>
<b>Subject:</b> Re: [scikit-learn] Why does sklearn require one-hot-encoding for categorical features? Can we have a "factor" data type?</font>
<div> </div>
</div>
<div><font size="2"><span style="font-size:11pt">
<div>On Thu, Apr 30, 2020 at 03:55:00PM -0400, C W wrote:<br>
> I've used R and Stata software, none needs such transformation. They have a<br>
> data type called "factors", which is different from "numeric".<br>
<br>
> My problem with OHE:<br>
> One-hot-encoding results in large number of features. This really blows up<br>
> quickly. And I have to fight curse of dimensionality with PCA reduction. That's<br>
> not cool!<br>
<br>
Most statistical models still not one-hot encoding behind the hood. So, R<br>
and stata do it too.<br>
<br>
Typically, tree-based models can be adapted to work directly on<br>
categorical data. Ours don't. It's work in progress.<br>
<br>
G<br>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fscikit-learn&data=02%7C01%7C%7Ce7aa6f99b7914a1f84b208d7ed430801%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637238744453345410&sdata=e3BfHB4v5VFteeZ0Zh3FJ9Wcz9KmkUwur5i8Reue3mc%3D&reserved=0" target="_blank" rel="noreferrer">https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Fscikit-learn&data=02%7C01%7C%7Ce7aa6f99b7914a1f84b208d7ed430801%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637238744453345410&sdata=e3BfHB4v5VFteeZ0Zh3FJ9Wcz9KmkUwur5i8Reue3mc%3D&reserved=0</a><br>
</div>
</span></font></div>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><span><div><div>Guillaume Lemaitre<br></div></div><div><span>Scikit-learn @ Inria Foundation</span><br><a href="https://glemaitre.github.io/" target="_blank" rel="noreferrer">https://glemaitre.github.io/</a></div></span></div></div></div></div></div></div></div></div></div></div></div></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>