[scikit-learn] LogisticRegression

[Eric J. Van der Velden]

Here I have added what I had programmed.

With sklearn's LogisticRegression(), how can I see the parameters it has
found after .fit() where the cost is minimal? I use the book of Geron about
scikit-learn and tensorflow and on page 137 he trains the model of petal
widths. I did the following:

    iris=datasets.load_iris()
    a1=iris['data'][:,3:]
    y=(iris['target']==2).astype(int)
    log_reg=LogisticRegression()
    log_reg.fit(a1,y)

    log_reg.coef_
    array([[2.61727777]])
    log_reg.intercept_
    array([-4.2209364])


I did the logistic regression myself with Gradient Descent or
Newton-Raphson as I learned from my Coursera course and respectively from
my book of Bishop. I used the Gradient Descent method like so:

    from sklearn import datasets
    iris=datasets.load_iris()
    a1=iris['data'][:,3:]
    A1=np.c_[np.ones((150,1)),a1]
    y=(iris['target']==2).astype(int).reshape(-1,1)
    lmda=1

    from scipy.special import expit

    def logreg_gd(w):
      z2=A1.dot(w)
      a2=expit(z2)
      delta2=a2-y
      w=w-(lmda/len(a1))*A1.T.dot(delta2)
      return w

    w=np.array([[0],[0]])
    for i in range(0,100000):
      w=logreg_gd(w)

    In [6219]: w
    Out[6219]:
    array([[-21.12563996],
           [ 12.94750716]])

I used Newton-Raphson like so, see Bishop page 207,

    from sklearn import datasets
    iris=datasets.load_iris()
    a1=iris['data'][:,3:]
    A1=np.c_[np.ones(len(a1)),a1]
    y=(iris['target']==2).astype(int).reshape(-1,1)

    def logreg_nr(w):
      z1=A1.dot(w)
      y=expit(z1)
      R=np.diag((y*(1-y))[:,0])
      H=A1.T.dot(R).dot(A1)
      tmp=A1.dot(w)-np.linalg.inv(R).dot(y-t)
      v=np.linalg.inv(H).dot(A1.T).dot(R).dot(tmp)
      return v

    w=np.array([[0],[0]])
    for i in range(0,10):
      w=logreg_nr(w)

    In [5149]: w
    Out[5149]:
    array([[-21.12563996],
           [ 12.94750716]])

Notice how much faster Newton-Raphson goes than Gradient Descent. But they
give the same result.

 How can I see which parameters LogisticRegression() found? And should I
give LogisticRegression other parameters?

On Sat, Jun 8, 2019 at 11:34 AM Eric J. Van der Velden <
ericjvandervelden at gmail.com> wrote:

> Hello,
>
> I am learning sklearn from my book of Geron. On page 137 he learns the
> model of petal widths.
>
> When I implements logistic regression myself as I learned from my Coursera
> course or from my book of Bishop I find that the following parameters are
> found where the cost function is minimal:
>
> In [6219]: w
> Out[6219]:
> array([[-21.12563996],
>        [ 12.94750716]])
>
> I used Gradient Descent and Newton-Raphson, both give the same answer.
>
> My question is: how can I see after fit() which parameters
> LogisticRegression() has found?
>
> One other question also: when I read the documentation page,
> https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression,
> I see a different cost function as I read in the books.
>
> Thanks.
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20190608/dc37fc86/attachment.html>