Package-level declarations

Gaussian Process for Regression.

Regression tree. A classification/regression tree can be learned by splitting the training set into subsets based on an attribute value test. This process is repeated on each derived subset in a recursive manner called recursive partitioning. The recursion is completed when the subset at a node all has the same value of the target variable, or when splitting no longer adds value to the predictions.

Gradient boosted regression trees.

Gaussian Process for Regression. A Gaussian process is a stochastic process whose realizations consist of random values associated with every point in a range of times (or of space) such that each such random variable has a normal distribution. Moreover, every finite collection of those random variables has a multivariate normal distribution.

Least absolute shrinkage and selection operator. The Lasso is a shrinkage and selection method for linear regression. It minimizes the usual sum of squared errors, with a bound on the sum of the absolute values of the coefficients (i.e. L₁-regularized). It has connections to soft-thresholding of wavelet coefficients, forward stage-wise regression, and boosting methods.

Fitting linear models (ordinary least squares). In linear regression, the model specification is that the dependent variable is a linear combination of the parameters (but need not be linear in the independent variables). The residual is the difference between the value of the dependent variable predicted by the model, and the true value of the dependent variable. Ordinary least squares obtains parameter estimates that minimize the sum of squared residuals, SSE (also denoted RSS).

Random forest for regression. Random forest is an ensemble classifier that consists of many decision trees and outputs the majority vote of individual trees. The method combines bagging idea and the random selection of features.

Radial basis function networks. A radial basis function network is an artificial neural network that uses radial basis functions as activation functions. It is a linear combination of radial basis functions. They are used in function approximation, time series prediction, and control.

Trains a Gaussian RBF network with k-means.

Ridge Regression. When the predictor variables are highly correlated amongst themselves, the coefficients of the resulting least squares fit may be very imprecise. By allowing a small amount of bias in the estimates, more reasonable coefficients may often be obtained. Ridge regression is one method to address these issues. Often, small amounts of bias lead to dramatic reductions in the variance of the estimated model coefficients. Ridge regression is such a technique which shrinks the regression coefficients by imposing a penalty on their size. Ridge regression was originally developed to overcome the singularity of the X'X matrix. This matrix is perturbed so as to make its determinant appreciably different from 0.

Support vector regression. Like SVM for classification, the model produced by SVR depends only on a subset of the training data, because the cost function ignores any training data close to the model prediction (within a threshold).