Class SessionOptions

java.lang.Object
smile.onnx.SessionOptions
All Implemented Interfaces:
AutoCloseable
public class SessionOptions extends Object implements AutoCloseable
Configuration options for an InferenceSession. Use the fluent builder methods to configure the session before passing this object to InferenceSession.create(String).

Instances of this class hold a native OrtSessionOptions pointer and must be closed after use (or used within a try-with-resources block). 

try (var opts = new SessionOptions()) {
    opts.setIntraOpNumThreads(4)
        .setGraphOptimizationLevel(GraphOptimizationLevel.ENABLE_ALL);
    try (var session = InferenceSession.create("model.onnx", opts)) {
        // ...
    }
}

  • Constructor Details

    • SessionOptions

      public SessionOptions()
      Creates a new SessionOptions with default settings.

  • Method Details

    • setIntraOpNumThreads

      public SessionOptions setIntraOpNumThreads(int numThreads)
      Sets the number of threads used to parallelize execution within a single operator (intra-op parallelism).

      The default value 0 lets ORT pick an appropriate number.

      Parameters:
      numThreads - the number of intra-op threads (0 = auto).
      Returns:
      this options object for chaining.

    • setInterOpNumThreads

      public SessionOptions setInterOpNumThreads(int numThreads)
      Sets the number of threads used to parallelize execution across independent operators (inter-op parallelism).

      The default value 0 lets ORT pick an appropriate number.

      Parameters:
      numThreads - the number of inter-op threads (0 = auto).
      Returns:
      this options object for chaining.

    • setGraphOptimizationLevel

      public SessionOptions setGraphOptimizationLevel(GraphOptimizationLevel level)
      Sets the graph optimization level.
      Parameters:
      level - the optimization level.
      Returns:
      this options object for chaining.

    • setExecutionMode

      public SessionOptions setExecutionMode(ExecutionMode mode)
      Sets the execution mode.
      Parameters:
      mode - the execution mode.
      Returns:
      this options object for chaining.

    • setLogId

      public SessionOptions setLogId(String logId)
      Sets the log identifier for this session's messages.
      Parameters:
      logId - the log identifier string.
      Returns:
      this options object for chaining.

    • setLogVerbosityLevel

      public SessionOptions setLogVerbosityLevel(int level)
      Sets the verbosity level of session log messages.
      Parameters:
      level - the logging verbosity level (higher = more verbose).
      Returns:
      this options object for chaining.

    • setLogSeverityLevel

      public SessionOptions setLogSeverityLevel(LoggingLevel level)
      Sets the minimum severity of log messages emitted by this session.
      Parameters:
      level - the minimum logging level.
      Returns:
      this options object for chaining.

    • setOptimizedModelFilePath

      public SessionOptions setOptimizedModelFilePath(String optimizedModelFilePath)
      Writes the optimized model to the given file path after session initialization.
      Parameters:
      optimizedModelFilePath - the output path for the optimized model.
      Returns:
      this options object for chaining.

    • enableCpuMemArena

      public SessionOptions enableCpuMemArena()
      Enables CPU memory arena allocation.
      Returns:
      this options object for chaining.

    • disableCpuMemArena

      public SessionOptions disableCpuMemArena()
      Disables CPU memory arena allocation.
      Returns:
      this options object for chaining.

    • enableMemPattern

      public SessionOptions enableMemPattern()
      Enables memory pattern optimization.
      Returns:
      this options object for chaining.

    • disableMemPattern

      public SessionOptions disableMemPattern()
      Disables memory pattern optimization.
      Returns:
      this options object for chaining.

    • enableProfiling

      public SessionOptions enableProfiling(String profileFilePrefix)
      Enables profiling. Profile data is written to the given file prefix.
      Parameters:
      profileFilePrefix - the file path prefix for profiling output.
      Returns:
      this options object for chaining.

    • disableProfiling

      public SessionOptions disableProfiling()
      Disables profiling.
      Returns:
      this options object for chaining.

    • appendCudaExecutionProvider

      public SessionOptions appendCudaExecutionProvider(int deviceId)
      Appends the CUDA execution provider (GPU device deviceId).
      Parameters:
      deviceId - the CUDA device index.
      Returns:
      this options object for chaining.

    • appendTensorRTExecutionProvider

      public SessionOptions appendTensorRTExecutionProvider(int deviceId)
      Appends the TensorRT execution provider (GPU device deviceId).
      Parameters:
      deviceId - the CUDA/TRT device index.
      Returns:
      this options object for chaining.

    • appendRocmExecutionProvider

      public SessionOptions appendRocmExecutionProvider(int deviceId)
      Appends the ROCM execution provider (AMD GPU device deviceId).
      Parameters:
      deviceId - the ROCM device index.
      Returns:
      this options object for chaining.

    • appendDirectMLExecutionProvider

      public SessionOptions appendDirectMLExecutionProvider(int deviceId)
      Appends the DirectML execution provider (Windows GPU, device deviceId).
      Parameters:
      deviceId - the DirectML device index.
      Returns:
      this options object for chaining.

    • addConfigEntry

      public SessionOptions addConfigEntry(String key, String value)
      Adds a session configuration entry as a key-value pair.
      Parameters:
      key - the configuration key.
      value - the configuration value.
      Returns:
      this options object for chaining.

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable