Class Avro

java.lang.Object
smile.io.Avro
public class Avro extends Object
Apache Avro is a data serialization system.

Avro provides rich data structures, a compact, fast, binary data format, a container file, to store persistent data, and remote procedure call (RPC).

Avro relies on schemas. When Avro data is stored in a file, its schema is stored with it. Avro schemas are defined with JSON.

  • Constructor Details

    • Avro

      public Avro(org.apache.avro.Schema schema)
      Constructor.
      Parameters:
      schema - the data schema.

    • Avro

      public Avro(InputStream schema) throws IOException
      Constructor.
      Parameters:
      schema - the input stream of schema.
      Throws:
      IOException - when fails to read the file.

    • Avro

      public Avro(Path schema) throws IOException
      Constructor.
      Parameters:
      schema - the path to Avro schema file.
      Throws:
      IOException - when fails to read the file.

  • Method Details

    • read

      public DataFrame read(Path path) throws IOException
      Reads an avro file.
      Parameters:
      path - the input file path.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.

    • read

      public DataFrame read(String path) throws IOException, URISyntaxException
      Reads an avro file.
      Parameters:
      path - the input file path.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.
      URISyntaxException - when the file path syntax is wrong.

    • read

      public DataFrame read(InputStream input, int limit) throws IOException
      Reads a limited number of records from an avro file.
      Parameters:
      input - the input stream of data file.
      limit - the number of records to read.
      Returns:
      the data frame.
      Throws:
      IOException - when fails to read the file.

    • toDataType

      public static DataType toDataType(org.apache.avro.Schema schema)
      Converts an avro type to smile data type.
      Parameters:
      schema - an avro schema.
      Returns:
      smile data type.

    • toStructField

      public static StructField toStructField(org.apache.avro.Schema.Field field)
      Converts an avro schema field to smile field.
      Parameters:
      field - an avro schema field.
      Returns:
      the struct field.

    • toStructType

      public static StructType toStructType(org.apache.avro.Schema schema)
      Converts an avro schema to smile schema.
      Parameters:
      schema - an avro schema.
      Returns:
      the struct type.