Avro Conversions

Provides parsers to read various formats as Avro records (currently JSON and XML), as well as tools to manipulate and/or describe schemas. The latter includes the conversion of XML Schema definitions (XSD) and JSON Schema into Avro schemas.

Documentation

The documentation is split into two parts. The impatient among us can start with the "Quickstart" section below. For the others, there is more elaborate documentation.

Quickstart

Want to jump right in? Here's a simple example to read a JSON schema, manipulate in into an Avro schema, and then parse a JSON file into the (different) Avro records:

import opwvhk.avro.SchemaManipulator;
import opwvhk.avro.json.JsonAsAvroParser;
import opwvhk.avro.util.NamingConvention;
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericRecord;

class Example {
  public static void main(String[] args) throws Exception {
    // Expected are 2 arguments:
    // * the location of a JSON schema file
    // * the location of a JSON file
    URL jsonSchemaLocation = new URL(args[0]);
    URL jsonFileLocation = new URL(args[1]);

    StringBuilder buffer = new StringBuilder();
    // Read an Avro schema; other methods can convert an XSD or JSON schema.
    Schema readSchema = SchemaManipulator.startFromJsonSchema(jsonSchemaLocation)
        .useSchemaNamingConvention(NamingConvention.PASCAL_CASE)
        .useFieldNamingConvention(NamingConvention.CAMEL_CASE)
        .renameField("newName", "path", "to", "field")
        .renameSchema("OldName", "NewName")
        .alsoDocumentAsMarkdownTable(buffer)
        .finish();
    // Print the schema as a Markdown table
    System.out.println(buffer);

    // Create a parser that reads JSON, validated according to the JSON schema, into Avro
    // records using a read schema.
    GenericData model = GenericData.get();
    JsonAsAvroParser parser = new JsonAsAvroParser(jsonSchemaLocation, readSchema, model);

    // The record type depends on the class generated by GenericData.get() (you can also use SpecificData or ReflectiveData).
    GenericRecord record = parser.parse(jsonFileLocation);
    System.out.println(record);
  }
}

Upgrade notes

From 1.x to 2.x

1. XML validation has been changed. Previously you could choose whether to validate per input. Now, whether inputs will be validated is a fixed setting per parser instance. See the constructor parameters for details.
2. Constructing a JSON parser can no longer throw a (library specific) GenerationException; this has become a runtime exception instead.

From 2.x to 3.x

1. Logging has changed: the library now uses the Java 9 Platform Logger (see Logging below)

Logging

This library uses the Java 9 Platform Logger (System.Logger) functionality, which is the new standard for libraries (it is a zero-dependency option allowing you to plug in your own logging framework). Dependencies however, are known to use SLF4J and Java Commons Logging (JCL).

Example configuration using Log4J

To log via Log4J, add these dependencies and configure Log4J:

• org.apache.logging.log4j:log4j-jpl
• org.apache.logging.log4j:log4j-slf4j2-impl
• org.apache.logging.log4j:log4j-core

JCL defers to Log4J if available (a builtin feature), and the dependencies add Log4J itself and autoconfigure the JDK and SLF4J to use Log4J.

Example configuration using Logback

To log via Logback, configure Log4J and add these dependencies:

• org.slf4j:slf4j-jdk-platform-logging
• ch.qos.logback:logback-classic

JCL defers to SLF4J if available (a builtin feature), and the dependencies add Logback (which natively supports SLF4J) and autoconfigure the JDK to use SLF4J.

Contributing ✨

Are you interested in contributing? These links may be of interest:

• Contributing Guidelines
• Good First Issues