public class RidgeRegression extends java.lang.Object implements Regression<double[]>, java.io.Serializable
X'X
becomes close to singular. As a result, the leastsquares estimate
becomes highly sensitive to random errors in the observed response Y
,
producing a large variance.
Ridge regression is one method to address these issues. In ridge regression,
the matrix X'X
is perturbed so as to make its determinant appreciably
different from 0.
Ridge regression is a kind of Tikhonov regularization, which is the most commonly used method of regularization of illposed problems. Ridge regression shrinks the regression coefficients by imposing a penalty on their size. By allowing a small amount of bias in the estimates, more reasonable coefficients may often be obtained. Often, small amounts of bias lead to dramatic reductions in the variance of the estimated model coefficients.
Another interpretation of ridge regression is available through Bayesian estimation. In this setting the belief that weight should be small is coded into a prior distribution.
The penalty term is unfair is the predictor variables are not on the same scale. Therefore, if we know that the variables are not measured in the same units, we typically scale the columns of X (to have sample variance 1), and then we perform ridge regression.
When including an intercept term in the regression, we usually leave
this coefficient unpenalized. Otherwise we could add some constant amount
to the vector y
, and this would not result in the same solution.
If we center the columns of X
, then the intercept estimate
ends up just being the mean of y
.
Ridge regression doesn’t set coefficients exactly to zero unless
λ = ∞
, in which case they’re all zero. Hence ridge regression cannot
perform variable selection, and even though it performs well in terms
of prediction accuracy, it does poorly in terms of offering a clear interpretation.
Modifier and Type  Class and Description 

static class 
RidgeRegression.Trainer
Trainer for ridge regression.

Constructor and Description 

RidgeRegression(double[][] x,
double[] y,
double lambda)
Constructor.

Modifier and Type  Method and Description 

double 
adjustedRSquared()
Returns adjusted R^{2} statistic.

double[] 
coefficients()
Returns the (scaled) linear coefficients.

int 
df()
Returns the degreeoffreedom of residual standard error.

double 
error()
Returns the residual standard error.

double 
ftest()
Returns the Fstatistic of goodnessoffit.

double 
intercept()
Returns the (centered) intercept.

double 
predict(double[] x)
Predicts the dependent variable of an instance.

double 
pvalue()
Returns the pvalue of goodnessoffit test.

double[] 
residuals()
Returns the residuals, that is response minus fitted values.

double 
RSquared()
Returns R^{2} statistic.

double 
RSS()
Returns the residual sum of squares.

double 
shrinkage()
Returns the shrinkage parameter.

java.lang.String 
toString() 
double[][] 
ttest()
Returns the ttest of the coefficients (without intercept).

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
predict
public RidgeRegression(double[][] x, double[] y, double lambda)
x
 a matrix containing the explanatory variables. NO NEED to include a constant column of 1s for bias.y
 the response values.lambda
 the shrinkage/regularization parameter. Large lambda means more shrinkage.
Choosing an appropriate value of lambda is important, and also difficult.public double[] coefficients()
public double intercept()
public double shrinkage()
public double predict(double[] x)
Regression
predict
in interface Regression<double[]>
x
 the instance.public double[][] ttest()
public double[] residuals()
public double RSS()
public double error()
public int df()
public double RSquared()
In the case of ordinary leastsquares regression, R^{2} increases as we increase the number of variables in the model (R^{2} will not decrease). This illustrates a drawback to one possible use of R^{2}, where one might try to include more variables in the model until "there is no more improvement". This leads to the alternative approach of looking at the adjusted R^{2}.
public double adjustedRSquared()
public double ftest()
public double pvalue()
public java.lang.String toString()
toString
in class java.lang.Object