Quick Start

$ bin/smile train
Smile 4.0.0
Usage: smile train [random_forest|gradient_boost|adaboost|cart|logistic|fisher|lda|qda|rda|mlp|svm|rbf|ols|lasso|ridge|elastic_net|gaussian_process] [options]

  --formula <class ~ .>    The model formula
  --data <file>            The training data file
  --test <file>            The optional test data file
  --model <file>           The model file to save
  --format <csv,header=true,delimiter=\t,comment=#,escape=\,quote=">
                           The data file format
  --kfold <value>          The k-fold cross validation
  --round <value>          The number of rounds of repeated cross validation
  --ensemble               Ensemble cross validation models
  --seed <value>           The random number generator seed
Command: random_forest [options]
Random Forest
  --regression             To train a regression model
  --trees <value>          The number of trees
  --mtry <value>           The number of features to train node split
  --split <GINI, ENTROPY, CLASSIFICATION_ERROR>
                           The split rule
  --max_depth <value>      The maximum tree depth
  --max_nodes <value>      The maximum number of leaf nodes
  --node_size <value>      The minimum leaf node size
  --sampling <value>       The sampling rate
  --class_weight <value>   The class weights
Command: gradient_boost [options]
Gradient Boosting
  --regression             To train a regression model
  --trees <value>          The number of trees
  --shrinkage <value>      The shrinkage parameter in (0, 1] controls the learning rate
  --max_depth <value>      The maximum tree depth
  --max_nodes <value>      The maximum number of leaf nodes
  --node_size <value>      The minimum leaf node size
  --sampling <value>       The sampling rate
Command: adaboost [options]
AdaBoost
  --trees <value>          The number of trees
  --max_depth <value>      The maximum tree depth
  --max_nodes <value>      The maximum number of leaf nodes
  --node_size <value>      The minimum leaf node size
Command: cart [options]
Classification and Regression Tree
  --regression             To train a regression model
  --split <GINI, ENTROPY, CLASSIFICATION_ERROR>
                           The split rule
  --max_depth <value>      The maximum tree depth
  --max_nodes <value>      The maximum number of leaf nodes
  --node_size <value>      The minimum leaf node size
Command: logistic [options]
Logistic Regression
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --lambda <value>         The regularization on linear weights
  --iterations <value>     The maximum number of iterations
  --tolerance <value>      The tolerance to stop iterations
Command: fisher [options]
Fisher Linear Discriminant
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --dimension <value>      The dimensionality of mapped space
  --tolerance <value>      The tolerance if a covariance matrix is singular
Command: lda [options]
Linear Discriminant Analysis
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --priori <value>         The priori probability of each class
  --tolerance <value>      The tolerance if a covariance matrix is singular
Command: qda [options]
Quadratic Discriminant Analysis
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --priori <value>         The priori probability of each class
  --tolerance <value>      The tolerance if a covariance matrix is singular
Command: rda [options]
Regularized Discriminant Analysis
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --alpha <value>          The regularization factor in [0, 1] allows a continuum of models between LDA and QDA
  --priori <value>         The priori probability of each class
  --tolerance <value>      The tolerance if a covariance matrix is singular
Command: mlp [options]
Multilayer Perceptron
  --regression             To train a regression model
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --layers <ReLU(100)|Sigmoid(30)>
                           The neural network layers
  --epochs <value>         The number of training epochs
  --mini_batch <value>     The split rule
  --learning_rate <0.01, linear(0.01, 10000, 0.001), piecewise(...), polynomial(...), inverse(...), exp(...)>
                           The learning rate schedule
  --momentum <value>       The momentum schedule
  --weight_decay <value>   The weight decay
  --clip_norm <value>      The gradient clipping norm
  --clip_value <value>     The gradient clipping value
  --rho <value>            RMSProp rho
  --epsilon <value>        RMSProp epsilon
Command: svm [options]
Support Vector Machine
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --kernel <value>         The kernel function
  --C <value>              The soft margin penalty parameter
  --epsilon <value>        The parameter of epsilon-insensitive hinge loss
  --ovr                    One vs Rest strategy for multiclass classification
  --ovo                    One vs One strategy for multiclass classification
  --tolerance <value>      The tolerance of convergence test
Command: rbf [options]
Radial Basis Function Network
  --regression             To train a regression model
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --neurons <value>        The number of neurons (radial basis functions)
  --normalize              Normalized RBF network
Command: ols [options]
Ordinary Least Squares
  --method <qr, svd>       The fitting method
  --stderr                 Compute the standard errors of the estimate of parameters.
  --recursive              Recursive least squares
Command: lasso [options]
LASSO - Least Absolute Shrinkage and Selection Operator
  --lambda <value>         The regularization on linear weights
  --iterations <value>     The maximum number of iterations
  --tolerance <value>      The tolerance to stop iterations (relative target duality gap)
Command: ridge [options]
Ridge Regression
  --lambda <value>         The regularization on linear weights
Command: elastic_net [options]
Elastic Net
  --lambda1 <value>        The L1 regularization on linear weights
  --lambda2 <value>        The L2 regularization on linear weights
  --iterations <value>     The maximum number of iterations
  --tolerance <value>      The tolerance to stop iterations (relative target duality gap)
Command: gaussian_process [options]
Gaussian Process Regression
  --transform <standardizer, winsor(0.01,0.99), minmax, MaxAbs, L1, L2, Linf>
                           The feature transformation
  --kernel <value>         The kernel function
  --noise <value>          The noise variance
  --normalize              Normalize the response variable
  --iterations <value>     The maximum number of HPO iterations
  --tolerance <value>      The stopping tolerance for HPO
          

$ bin/smile train random_forest --data data/weka/iris.arff --formula "class ~ ." --model iris_random_forest.model
[main] INFO smile.io.Arff - Read ARFF relation iris
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 88.89%
[ForkJoinPool.commonPool-worker-2] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 95.35%
[ForkJoinPool.commonPool-worker-1] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 96.67%
...
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 92.73%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 94.44%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 92.98%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 92.31%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 95.00%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 96.30%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 89.47%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 92.98%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 92.45%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 96.30%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 89.83%
[ForkJoinPool.commonPool-worker-3] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 90.57%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 94.64%
[main] INFO smile.classification.RandomForest - Decision tree OOB accuracy: 97.92%
Training metrics: {
  fit time: 191.678 ms,
  score time: 17.059 ms,
  validation data size: 150,
  error: 6,
  accuracy: 96.00%,
  cross entropy: 0.1316
}
  

$ bin/smile predict --model iris_random_forest.model --data data/weka/iris.arff --probability
0 [0.9599005925578108, 0.0205362892965633, 0.01956311814562593]
0 [0.9591845549270528, 0.020962057229598156, 0.01985338784334913]
0 [0.959796981740008, 0.02072034636162096, 0.019482671898371082]
0 [0.959796981740008, 0.02072034636162096, 0.019482671898371082]
0 [0.9598769149423004, 0.020548128104318476, 0.0195749569533811]
0 [0.9583053192555514, 0.021164037082801724, 0.020530643661646933]
0 [0.9598769149423004, 0.020548128104318476, 0.0195749569533811]
0 [0.9598769149423004, 0.020548128104318476, 0.0195749569533811]
0 [0.9591845549270528, 0.020962057229598156, 0.01985338784334913]
0 [0.959796981740008, 0.02072034636162096, 0.019482671898371082]
0 [0.9583053192555514, 0.021164037082801724, 0.020530643661646933]
0 [0.9598769149423004, 0.020548128104318476, 0.0195749569533811]
0 [0.9591845549270528, 0.020962057229598156, 0.01985338784334913]
0 [0.9591845549270528, 0.020962057229598156, 0.01985338784334913]
0 [0.9027507265438325, 0.07089506779135216, 0.026354205664815347]
0 [0.911215084168623, 0.0632270620116508, 0.025557853819726185]
0 [0.9583053192555514, 0.021164037082801724, 0.020530643661646933]
0 [0.9599005925578108, 0.0205362892965633, 0.01956311814562593]
0 [0.911215084168623, 0.0632270620116508, 0.025557853819726185]
...
  

$ bin/smile serve --model iris_random_forest.model --probability
[smile-akka.actor.default-dispatcher-4] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
Smile online at http://localhost:8080/v1/infer
Press RETURN to stop...
  

$ curl -X POST http://localhost:8080/v1/infer -H "Content-Type: application/json" \
        -d '{
          "sepallength": 5.1,
          "sepalwidth": 3.5,
          "petallength": 1.4,
          "petalwidth": 0.2
        }'
[{"class":0,"probability":[0.9599005925578108,0.0205362892965633,0.01956311814562593]}]
  

$ curl -X POST http://localhost:8080/v1/infer -H "Content-Type: application/json" \
        -d '{"sepallength": 5.1, "sepalwidth": 3.5, "petallength": 1.4,"petalwidth": 0.2}
          {"sepallength": 6.3, "sepalwidth": 3.3, "petallength": 6.0,"petalwidth": 2.5}'
[{"class":0,"probability":[0.9599005925578108,0.0205362892965633,0.01956311814562593]},
 {"class":2,"probability":[0.023727657781681327,0.051035220743102516,0.9252371214752161]}]
  

$ for i in {1..1000}; do tail -n 153 data/weka/iris.arff | head -n 150 >> iris.txt; done
$ cat iris.txt | curl -H "Content-Type: text/csv" -X POST --data-binary @- http://localhost:8080/v1/infer?format=csv
  

    import $ivy.`com.github.haifengl::smile-scala:4.0.0`

    import scala.language.postfixOps
    import org.apache.commons.csv.CSVFormat
    import smile._
    import smile.util._
    import smile.math._
    import smile.math.MathEx.{log2, logistic, factorial, lfactorial, choose, lchoose, random, randomInt, permutate, c, cbind, rbind, sum, mean, median, q1, q3, `var` => variance, sd, mad, min, max, whichMin, whichMax, unique, dot, distance, pdist, KullbackLeiblerDivergence => kld, JensenShannonDivergence => jsd, cov, cor, spearman, kendall, norm, norm1, norm2, normInf, standardize, normalize, scale, unitize, unitize1, unitize2, root}
    import smile.math.distance._
    import smile.math.kernel._
    import smile.math.matrix._
    import smile.math.matrix.Matrix._
    import smile.math.rbf._
    import smile.stat.distribution._
    import smile.data._
    import smile.data.formula._
    import smile.data.measure._
    import smile.data.`type`._
    import smile.json._
    import smile.interpolation._
    import smile.validation._
    import smile.association._
    import smile.base.cart.SplitRule
    import smile.base.mlp._
    import smile.base.rbf.RBF
    import smile.classification._
    import smile.regression.{ols, ridge, lasso, svr, gpr}
    import smile.feature._
    import smile.clustering._
    import smile.vq._
    import smile.manifold._
    import smile.mds._
    import smile.sequence._
    import smile.projection._
    import smile.nlp._
    import smile.wavelet._
    

    import smile.plot.swing._
    import smile.plot.show
    import smile.plot.Render._
    

    import smile.plot.vega._
    import smile.plot.show
    import smile.plot.Render._
    

    import smile.io.*;
    var weather = Read.arff("data/weka/weather.nominal.arff");
          

    import org.apache.commons.csv.CSVFormat;

    var format = CSVFormat.DEFAULT.withDelimiter(' ');
    var zipTrain = Read.csv("data/usps/zip.train", format);
    var zipTest = Read.csv("data/usps/zip.test", format);
          

    import smile.classification.*;
    import smile.data.formula.Formula;

    var formula = Formula.lhs("V1");
    
    var prop = new java.util.Properties();
    prop.setProperty("smile.random.forest.trees", "200");
    var forest = RandomForest.fit(formula, zipTrain, prop);
    System.out.println(forest.metrics());
          

    var x = formula.x(zipTrain).toArray();
    var y = formula.y(zipTrain).toIntArray();
    var testx = formula.x(zipTest).toArray();
    var testy = formula.y(zipTest).toIntArray();
          

    import smile.math.kernel.GaussianKernel;
    import smile.validation.*;

    var kernel = new GaussianKernel(8.0);
    var svm = OneVersusOne.fit(x, y, (x, y) -> SVM.fit(x, y, kernel, 5, 1E-3));
    var pred = svm.predict(testx);
    System.out.format("Accuracy = %.2f%%%n", (100.0 * Accuracy.of(testy, pred)));
    System.out.format("Confusion Matrix: %s%n", ConfusionMatrix.of(testy, pred));
          

    import smile.base.mlp.Layer;
    import smile.base.mlp.OutputFunction;
    import smile.classification.MLP;
    import smile.math.MathEx;
    
    var net = new MLP(Layer.input(256),
      Layer.sigmoid(768),
      Layer.sigmoid(192),
      Layer.sigmoid(30),
      Layer.mle(10, OutputFunction.SIGMOID)
    );

    net.setLearningRate(TimeFunction.linear(0.01, 20000, 0.001));
    
    for (int epoch = 0; epoch < 10; epoch++) {
      System.out.format("----- epoch %d -----%n", epoch);
      for (int i : MathEx.permutate(x.length)) {
        net.update(x[i], y[i]);
      }
      var prediction = net.predict(testx);
      System.out.format("Accuracy = %.2f%%%n", (100.0 * Accuracy.of(testy, prediction)));
    }
          

    var posteriori = new double[10];
    forest.predict(zipTest.get(0), posteriori);
    svm.predict(testx[0]);
    net.predict(testx[0], posteriori);