smile.classification

Class LogisticRegression

• java.lang.Object
• smile.classification.LogisticRegression
• All Implemented Interfaces:
java.io.Serializable, java.util.function.ToDoubleFunction<double[]>, java.util.function.ToIntFunction<double[]>, Classifier<double[]>, OnlineClassifier<double[]>, SoftClassifier<double[]>
Direct Known Subclasses:
LogisticRegression.Binomial, LogisticRegression.Multinomial

```public abstract class LogisticRegression
extends java.lang.Object
implements SoftClassifier<double[]>, OnlineClassifier<double[]>```
Logistic regression. Logistic regression (logit model) is a generalized linear model used for binomial regression. Logistic regression applies maximum likelihood estimation after transforming the dependent into a logit variable. A logit is the natural log of the odds of the dependent equaling a certain value or not (usually 1 in binary logistic models, the highest value in multinomial models). In this way, logistic regression estimates the odds of a certain event (value) occurring.

Goodness-of-fit tests such as the likelihood ratio test are available as indicators of model appropriateness, as is the Wald statistic to test the significance of individual independent variables.

Logistic regression has many analogies to ordinary least squares (OLS) regression. Unlike OLS regression, however, logistic regression does not assume linearity of relationship between the raw values of the independent variables and the dependent, does not require normally distributed variables, does not assume homoscedasticity, and in general has less stringent requirements.

Compared with linear discriminant analysis, logistic regression has several advantages:

• It is more robust: the independent variables don't have to be normally distributed, or have equal variance in each group
• It does not assume a linear relationship between the independent variables and dependent variable.
• It may handle nonlinear effects since one can add explicit interaction and power terms.
However, it requires much more data to achieve stable, meaningful results.

Logistic regression also has strong connections with neural network and maximum entropy modeling. For example, binary logistic regression is equivalent to a one-layer, single-output neural network with a logistic activation function trained under log loss. Similarly, multinomial logistic regression is equivalent to a one-layer, softmax-output neural network.

Logistic regression estimation also obeys the maximum entropy principle, and thus logistic regression is sometimes called "maximum entropy modeling", and the resulting classifier the "maximum entropy classifier".

`GLM`, `MLP`, `Maxent`, `LDA`, Serialized Form
• Nested Class Summary

Nested Classes
Modifier and Type Class and Description
`static class ` `LogisticRegression.Binomial`
Binomial logistic regression.
`static class ` `LogisticRegression.Multinomial`
Multinomial logistic regression.
• Constructor Summary

Constructors
Constructor and Description
```LogisticRegression(int p, double L, double lambda, IntSet labels)```
Constructor.
• Method Summary

All Methods
Modifier and Type Method and Description
`double` `AIC()`
Returns the AIC score.
`static LogisticRegression.Binomial` ```binomial(double[][] x, int[] y)```
Fits binomial logistic regression.
`static LogisticRegression.Binomial` ```binomial(double[][] x, int[] y, double lambda, double tol, int maxIter)```
Fits binomial logistic regression.
`static LogisticRegression.Binomial` ```binomial(double[][] x, int[] y, java.util.Properties prop)```
Fits binomial logistic regression.
`static LogisticRegression.Binomial` ```binomial(Formula formula, DataFrame data)```
Fits binomial logistic regression.
`static LogisticRegression.Binomial` ```binomial(Formula formula, DataFrame data, java.util.Properties prop)```
Fits binomial logistic regression.
`static LogisticRegression` ```fit(double[][] x, int[] y)```
Fits logistic regression.
`static LogisticRegression` ```fit(double[][] x, int[] y, double lambda, double tol, int maxIter)```
Fits logistic regression.
`static LogisticRegression` ```fit(double[][] x, int[] y, java.util.Properties prop)```
Fits logistic regression.
`static LogisticRegression` ```fit(Formula formula, DataFrame data)```
Fits logistic regression.
`static LogisticRegression` ```fit(Formula formula, DataFrame data, java.util.Properties prop)```
Fits logistic regression.
`double` `getLearningRate()`
Returns the learning rate of stochastic gradient descent.
`double` `loglikelihood()`
Returns the log-likelihood of model.
`static LogisticRegression.Multinomial` ```multinomial(double[][] x, int[] y)```
Fits multinomial logistic regression.
`static LogisticRegression.Multinomial` ```multinomial(double[][] x, int[] y, double lambda, double tol, int maxIter)```
Fits multinomial logistic regression.
`static LogisticRegression.Multinomial` ```multinomial(double[][] x, int[] y, java.util.Properties prop)```
Fits multinomial logistic regression.
`static LogisticRegression.Multinomial` ```multinomial(Formula formula, DataFrame data)```
Fits multinomial logistic regression.
`static LogisticRegression.Multinomial` ```multinomial(Formula formula, DataFrame data, java.util.Properties prop)```
Fits multinomial logistic regression.
`void` `setLearningRate(double rate)`
Sets the learning rate of stochastic gradient descent.
• Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`
• Methods inherited from interface smile.classification.SoftClassifier

`predict`
• Methods inherited from interface smile.classification.OnlineClassifier

`update, update`
• Methods inherited from interface smile.classification.Classifier

`applyAsDouble, applyAsInt, f, predict, predict`
• Constructor Detail

• LogisticRegression

```public LogisticRegression(int p,
double L,
double lambda,
IntSet labels)```
Constructor.
Parameters:
`p` - the dimension of input data.
`L` - the log-likelihood of learned model.
`lambda` - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.
`labels` - class labels
• Method Detail

• binomial

```public static LogisticRegression.Binomial binomial(Formula formula,
DataFrame data)```
Fits binomial logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• binomial

```public static LogisticRegression.Binomial binomial(Formula formula,
DataFrame data,
java.util.Properties prop)```
Fits binomial logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• binomial

```public static LogisticRegression.Binomial binomial(double[][] x,
int[] y)```
Fits binomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• binomial

```public static LogisticRegression.Binomial binomial(double[][] x,
int[] y,
java.util.Properties prop)```
Fits binomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• binomial

```public static LogisticRegression.Binomial binomial(double[][] x,
int[] y,
double lambda,
double tol,
int maxIter)```
Fits binomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
`lambda` - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.
`tol` - the tolerance for stopping iterations.
`maxIter` - the maximum number of iterations.
• multinomial

```public static LogisticRegression.Multinomial multinomial(Formula formula,
DataFrame data)```
Fits multinomial logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• multinomial

```public static LogisticRegression.Multinomial multinomial(Formula formula,
DataFrame data,
java.util.Properties prop)```
Fits multinomial logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• multinomial

```public static LogisticRegression.Multinomial multinomial(double[][] x,
int[] y)```
Fits multinomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• multinomial

```public static LogisticRegression.Multinomial multinomial(double[][] x,
int[] y,
java.util.Properties prop)```
Fits multinomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• multinomial

```public static LogisticRegression.Multinomial multinomial(double[][] x,
int[] y,
double lambda,
double tol,
int maxIter)```
Fits multinomial logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
`lambda` - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.
`tol` - the tolerance for stopping iterations.
`maxIter` - the maximum number of iterations.
• fit

```public static LogisticRegression fit(Formula formula,
DataFrame data)```
Fits logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• fit

```public static LogisticRegression fit(Formula formula,
DataFrame data,
java.util.Properties prop)```
Fits logistic regression.
Parameters:
`formula` - a symbolic description of the model to be fitted.
`data` - the data frame of the explanatory and response variables.
• fit

```public static LogisticRegression fit(double[][] x,
int[] y)```
Fits logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• fit

```public static LogisticRegression fit(double[][] x,
int[] y,
java.util.Properties prop)```
Fits logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
• fit

```public static LogisticRegression fit(double[][] x,
int[] y,
double lambda,
double tol,
int maxIter)```
Fits logistic regression.
Parameters:
`x` - training samples.
`y` - training labels.
`lambda` - λ > 0 gives a "regularized" estimate of linear weights which often has superior generalization performance, especially when the dimensionality is high.
`tol` - the tolerance for stopping iterations.
`maxIter` - the maximum number of iterations.
• setLearningRate

`public void setLearningRate(double rate)`
Sets the learning rate of stochastic gradient descent. It is a good practice to adapt the learning rate for different data sizes. For example, it is typical to set the learning rate to eta/n, where eta is in [0.1, 0.3] and n is the size of the training data.
Parameters:
`rate` - the learning rate.
• getLearningRate

`public double getLearningRate()`
Returns the learning rate of stochastic gradient descent.
• loglikelihood

`public double loglikelihood()`
Returns the log-likelihood of model.
• AIC

`public double AIC()`
Returns the AIC score.