Class Hyperparameters

java.lang.Object
smile.hpo.Hyperparameters
public class Hyperparameters extends Object
Hyperparameter configuration. A hyperparameter is a parameter whose value is set before the learning process begins. By contrast, the values of other parameters are derived via training.

Hyperparameters can be classified as model hyperparameters, that cannot be inferred while fitting the machine to the training set because they refer to the model selection task, or algorithm hyperparameters, that in principle have no influence on the performance of the model but affect the speed and quality of the learning process. For example, the topology and size of a neural network are model hyperparameters, while learning rate and mini-batch size are algorithm hyperparameters.

The below example shows how to tune the hyperparameters of random forest using grid search. 


   import smile.io.*;
   import smile.data.formula.Formula;
   import smile.validation.*;
   import smile.classification.RandomForest;

   var hp = new Hyperparameters()
       .add("smile.random.forest.trees", 100) // a fixed value
       .add("smile.random.forest.mtry", new int[] {2, 3, 4}) // discrete choices
       .add("smile.random.forest.max.nodes", 100, 500, 50); // range [100, 500] step 50

   var train = Read.arff("data/weka/segment-challenge.arff");
   var test = Read.arff("data/weka/segment-test.arff");
   var formula = Formula.lhs("class");
   var testy = formula.y(test).toIntArray();

   hp.grid().forEach(prop -> {
       var model = RandomForest.fit(formula, train, prop);
       var pred = model.predict(test);
       System.out.println(prop);
       System.out.format("Accuracy = %.2f%%%n", (100.0 * Accuracy.of(testy, pred)));
       System.out.println(ConfusionMatrix.of(testy, pred));
   });

   // Random search — remember to limit the infinite stream:
   hp.random().limit(50).forEach(prop -> { ... });

  • Constructor Details

    • Hyperparameters

      public Hyperparameters()
      Constructor.

  • Method Details

    • add

      public Hyperparameters add(String name, int value)
      Adds a parameter with a fixed integer value.
      Parameters:
      name - the parameter name.
      value - a fixed value of the parameter.
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, double value)
      Adds a parameter with a fixed double value.
      Parameters:
      name - the parameter name.
      value - a fixed value of the parameter.
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, String value)
      Adds a parameter with a fixed string value.
      Parameters:
      name - the parameter name.
      value - a fixed value of the parameter.
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, int[] values)
      Adds a parameter with a discrete set of integer choices.
      Parameters:
      name - the parameter name.
      values - a non-empty array of candidate values.
      Returns:
      this object.
      Throws:
      IllegalArgumentException - if values is empty.

    • add

      public Hyperparameters add(String name, double[] values)
      Adds a parameter with a discrete set of double choices.
      Parameters:
      name - the parameter name.
      values - a non-empty array of candidate values.
      Returns:
      this object.
      Throws:
      IllegalArgumentException - if values is empty.

    • add

      public Hyperparameters add(String name, String[] values)
      Adds a parameter with a discrete set of string choices.
      Parameters:
      name - the parameter name.
      values - a non-empty array of candidate values.
      Returns:
      this object.
      Throws:
      IllegalArgumentException - if values is empty.

    • add

      public Hyperparameters add(String name, int start, int end)
      Adds an integer parameter with an auto-stepped range. The step defaults to max(1, (end-start)/10).
      Parameters:
      name - the parameter name.
      start - the start of value range (inclusive).
      end - the end of value range (inclusive upper bound for steps).
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, int start, int end, int step)
      Adds an integer parameter with an explicit step range.
      Parameters:
      name - the parameter name.
      start - the start of value range (inclusive).
      end - the end of value range (inclusive upper bound for steps).
      step - the step size (must be positive).
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, double start, double end)
      Adds a double parameter with an auto-stepped range. The step defaults to (end-start)/10.
      Parameters:
      name - the parameter name.
      start - the start of value range (inclusive).
      end - the end of value range (inclusive upper bound for steps).
      Returns:
      this object.

    • add

      public Hyperparameters add(String name, double start, double end, double step)
      Adds a double parameter with an explicit step range.
      Parameters:
      name - the parameter name.
      start - the start of value range (inclusive).
      end - the end of value range (inclusive upper bound for steps).
      step - the step size (must be positive).
      Returns:
      this object.

    • remove

      public Hyperparameters remove(String name)
      Removes a previously registered parameter. Has no effect if the parameter was not registered.
      Parameters:
      name - the parameter name.
      Returns:
      this object.

    • clear

      public Hyperparameters clear()
      Removes all registered parameters.
      Returns:
      this object.

    • size

      public int size()
      Returns the number of registered parameters.
      Returns:
      the number of registered parameters.

    • random

      public Stream<Properties> random()
      Generates an infinite stream of randomly sampled hyperparameter configurations. The caller must apply Stream.limit(long) before collecting, e.g.: 
      hp.random().limit(100).forEach(this::evaluate);

      For Hyperparameters.IntRange and Hyperparameters.DoubleRange parameters the step field is ignored; values are sampled uniformly from [start, end).

      Returns:
      an infinite stream of hyperparameter configurations.
      Throws:
      IllegalStateException - if no parameters have been registered.

    • random

      public Stream<Properties> random(int n)
      Generates a finite stream of n randomly sampled hyperparameter configurations. This is a convenience alternative to random().limit(n).
      Parameters:
      n - the number of configurations to sample (must be positive).
      Returns:
      a stream of n randomly sampled configurations.
      Throws:
      IllegalArgumentException - if n <= 0.
      IllegalStateException - if no parameters have been registered.

    • grid

      public Stream<Properties> grid()
      Generates a finite stream of all hyperparameter configurations for grid (exhaustive) search. The total number of configurations is the Cartesian product of all registered parameter value lists — be cautious of combinatorial explosion.

      Parameters are enumerated in insertion order.

      Returns:
      a finite stream of all grid-search configurations.
      Throws:
      IllegalStateException - if no parameters have been registered.