app-encryption

Asherah - Java

Application level envelope encryption SDK for Java with support for cloud-agnostic data storage and key management.

• Installation
• Quick Start
• How to Use Asherah
  • Define the Metastore
  • Define the Key Management Service
  • Define the Crypto Policy
    • (Optional) Enable Session Caching
  • (Optional) Enable Metrics
  • Build a Session Factory
  • Performing Cryptographic Operations
• Deployment Notes
  • Handling read-only Docker containers
• Development Notes

Installation

You can include Asherah in Java projects projects using Maven

The Maven group ID is com.godaddy.asherah, and the artifact ID is appencryption.

You can specify the current release of Asherah as a project dependency using the following configuration:

<dependencies>
  <dependency>
    <groupId>com.godaddy.asherah</groupId>
    <artifactId>appencryption</artifactId>
    <version>0.3.0</version>
  </dependency>
</dependencies>

Quick Start

// Create a session factory. The builder steps used below are for testing only.
try (SessionFactory sessionFactory = SessionFactory.newBuilder("some_product", "some_service")
    .withInMemoryMetastore()
    .withNeverExpiredCryptoPolicy()
    .withStaticKeyManagementService("thisIsAStaticMasterKeyForTesting")
    .build()) {

  // Now create a cryptographic session for a partition.
  try (Session<byte[], byte[]> sessionBytes = sessionFactory.getSessionBytes("some_partition")) {

    // Now encrypt some data
    String originalPayloadString = "mysupersecretpayload";
    byte[] dataRowRecordBytes = sessionBytes.encrypt(originalPayloadString.getBytes(StandardCharsets.UTF_8));

    // Decrypt the data
    String decryptedPayloadString = new String(sessionBytes.decrypt(dataRowRecordBytes), StandardCharsets.UTF_8);
  }
}

A more extensive example is the Reference Application, which will evolve along with the SDK.

How to Use Asherah

Before you can start encrypting data, you need to define Asherah's required pluggable components. Below we show how to build the various options for each component.

Define the Metastore

Detailed information about the Metastore, including any provisioning steps, can be found here.

RDBMS Metastore

Asherah can connect to a relational database by accepting a JDBC DataSource for connection handling.

// Create / retrieve a DataSource from your connection pool
DataSource dataSource = ...;

// Build the JDBC Metastore
Metastore jdbcMetastore = JdbcMetastoreImpl.newBuilder(dataSource).build();

DynamoDB Metastore

For simplicity, the DynamoDB implementation uses the builder pattern to enable configuration changes.

To obtain an instance of the builder, use the static factory method newBuilder.

DynamoDbMetastoreImpl.newBuilder();

Once you have a builder, you can either use the withXXX setter methods to configure the metastore properties or simply build the metastore by calling the build method.

• withKeySuffix: Specifies whether key suffix should be enabled for DynamoDB. This is required to enable Global Tables.
• withTableName: Specifies the name of the DynamoDb table.
• withRegion: Specifies the region for the AWS DynamoDb client.
• withEndPointConfiguration: Adds an EndPoint configuration to the AWS DynamoDb client.
• withClientOverride: Specifies a custom AWS DynamoDb client. Region and endpoint configuration will be ignored if custom client is provided.

Below is an example of a DynamoDB metastore that uses a Global Table named TestTable

Metastore dynamoDbMetastore = DynamoDbMetastoreImpl.newBuilder("us-west-2")
      .withTableName("TestTable")
      .withKeySuffix()
      .build();

In-memory Metastore (FOR TESTING ONLY)

Metastore<JSONObject> metastore = new InMemoryMetastoreImpl<>();

Define the Key Management Service

Detailed information about the Key Management Service can be found here.

AWS KMS

// Create a map of region and ARN pairs that will all be used when encrypting a System Key
Map<String, String> regionMap = ImmutableMap.of("us-east-1", "arn_of_us-east-1",
    "us-east-2", "arn_of_us-east-2",
    ...);

// Build the Key Management Service using the region map and your preferred (usually current) region
KeyManagementService keyManagementService = AwsKeyManagementServiceImpl.newBuilder(regionMap, "us-east-1").build();

It is possible to specify AWS KMS client factory to be used, instead of default one:

// Define AWS KMS client factory
AwsKmsClientFactory awsKmsClientFactory = ...;

// Build the Key Management Service using the region map and your preferred (usually current) region and custom AWS KMS client factory
KeyManagementService keyManagementService = AwsKeyManagementServiceImpl.newBuilder(regionMap, "us-east-1")
    .withAwsKmsClientFactory(awsKmsClientFactory)
    .build();

Static KMS (FOR TESTING ONLY)

KeyManagementService keyManagementService = new StaticKeyManagementServiceImpl("thisIsAStaticMasterKeyForTesting");

Define the Crypto Policy

Detailed information about Crypto Policy can be found here. The Crypto Policy's effect on key caching is explained here.

Basic Expiring Crypto Policy

CryptoPolicy basicExpiringCryptoPolicy = BasicExpiringCryptoPolicy.newBuilder()
    .withKeyExpirationDays(90)
    .withRevokeCheckMinutes(60)
    .build();

(Optional) Enable Session Caching

Session caching is disabled by default. Enabling it is primarily useful if you are working with stateless workloads and the shared session can't be used by the calling app.

To enable session caching, simply use the optional builder step withCanCacheSessions(true) when building a crypto policy.

CryptoPolicy basicExpiringCryptoPolicy = BasicExpiringCryptoPolicy.newBuilder()
    .withKeyExpirationDays(90)
    .withRevokeCheckMinutes(60)
    .withCanCacheSessions(true)    // Enable session cache
    .withSessionCacheMaxSize(200)    // Define the number of maximum sessions to cache
    .withSessionCacheExpireMinutes(5)    // Evict the session from cache after some minutes
    .build();

Never Expired Crypto Policy (FOR TESTING ONLY)

CryptoPolicy neverExpiredCryptoPolicy = new NeverExpiredCryptoPolicy();

(Optional) Enable Metrics

Asherah's Java implementation uses Micrometer for metrics, which are disabled by default. All metrics generated by this SDK use the global registry and use a prefix defined by MetricsUtil.AEL_METRICS_PREFIX (ael as of this writing). If metrics are left disabled, we rely on Micrometer's deny filtering.

To enable metrics generation, simply use the final optional builder step withMetricsEnabled() when building a session factory:

The following metrics are available:

• ael.drr.decrypt: Total time spent on all operations that were needed to decrypt.
• ael.drr.encrypt: Total time spent on all operations that were needed to encrypt.
• ael.kms.aws.decrypt.<region>: Time spent on decrypting the region-specific keys.
• ael.kms.aws.decryptkey: Total time spend in decrypting the key which would include the region-specific decrypt calls in case of transient failures.
• ael.kms.aws.encrypt.<region>: Time spent on data key plain text encryption for each region.
• ael.kms.aws.encryptkey: Total time spent in encrypting the key which would include the region-specific generatedDataKey and parallel encrypt calls.
• ael.kms.aws.generatedatakey.<region>: Time spent to generate the first data key which is then encrypted in remaining regions.
• ael.metastore.jdbc.load: Time spent to load a record from jdbc metastore.
• ael.metastore.jdbc.loadlatest: Time spent to get the latest record from jdbc metastore.
• ael.metastore.jdbc.store: Time spent to store a record into jdbc metastore.
• ael.metastore.dynamodb.load: Time spent to load a record from DynamoDB metastore.
• ael.metastore.dynamodb.loadlatest: Time spent to get the latest record from DynamoDB metastore.
• ael.metastore.dynamodb.store: Time spent to store a record into DynamoDB metastore.

Build a Session Factory

A session factory can now be built using the components we defined above.

SessionFactory sessionFactory = SessionFactory.newBuilder("some_product", "some_service")
  .withMetastore(metastore)
  .withCryptoPolicy(policy)
  .withKeyManagementService(keyManagementService)
  .withMetricsEnabled() // optional
  .build();

NOTE: We recommend that every service have its own session factory, preferably as a singleton instance within the service. This will allow you to leverage caching and minimize resource usage. Always remember to close the session factory before exiting the service to ensure that all resources held by the factory, including the cache, are disposed of properly.

Performing Cryptographic Operations

Create a Session to be used for cryptographic operations.

Session<byte[], byte[]> sessionBytes = sessionFactory.getSessionBytes("some_user");

The different usage styles are explained below.

NOTE: Remember to close the session after all cryptographic operations to dispose of associated resources.

Plain Encrypt/Decrypt Style

This usage style is similar to common encryption utilities where payloads are simply encrypted and decrypted, and it is completely up to the calling application for storage responsibility.

String originalPayloadString = "mysupersecretpayload";

// encrypt the payload
byte[] dataRowRecordBytes = sessionBytes.encrypt(originalPayloadString.getBytes(StandardCharsets.UTF_8));

// decrypt the payload
String decryptedPayloadString = new String(sessionBytes.decrypt(dataRowRecordBytes), StandardCharsets.UTF_8);

Custom Persistence via Store/Load methods

Asherah supports a key-value/document storage model. A Session can accept a Persistence implementation and hook into its store and load calls.

An example HashMap-backed Persistence implementation:

Persistence dataPersistence = new Persistence<JSONObject>() {

  Map<String, JSONObject> mapPersistence = new HashMap<>();

  @Override
  public Optional<JSONObject> load(String key) {
    return Optional.ofNullable(mapPersistence.get(key));
  }

  @Override
  public void store(String key, JSONObject value) {
    mapPersistence.put(key, value);
  }
};

An example end-to-end use of the store and load calls:

// Encrypts the payload, stores it in the dataPersistence and returns a look up key
String persistenceKey = sessionJson.store(originalPayload.toJsonObject(), dataPersistence);

// Uses the persistenceKey to look-up the payload in the dataPersistence, decrypts the payload if any and then returns it
Optional<JSONObject> payload = sessionJson.load(persistenceKey, dataPersistence);

Deployment Notes

Handling read-only Docker containers

SecureMemory currently uses JNA for native calls. The default behavior of JNA is to unpack the native libraries from the jar to a temp folder, which can fail in a read-only container. The native library can instead be installed as a part of the container. The general steps are:

1. Install the native JNA package
2. Specify -Djna.nounpack=true so the container never attempts to unpack bundled native libraries from the JNA jar.

The following are distro-specific notes that we know about:

• Alpine

  • The native package is java-jna-native.

• Debian

  • The native package, libjna-jni, needs to be installed from the Debian testing repo as the current base image is not compatible with AEL's JNA version. The example Debian Dockerfile listed below adds the testing repo before installing this package and is then removed.
  • Add the property -Djna.boot.library.name=jnidispatch.system in java exec as Debian package contains an extra ".system" in the library name.

• Ubuntu

  • The native package is libjna-jni.
  • Add the property -Djna.boot.library.name=jnidispatch.system in java exec as Ubuntu package contains an extra ".system" in the library name.
  • If using the adoptopenjdk/openjdk base image, we need to add additional directories in the default library path using -Djna.boot.library.path=/usr/lib/x86_64-linux-gnu/jni/

Our test app repo's Dockerfiles can be used for reference: Alpine, Debian and Ubuntu (uses AdoptOpenJDK)

Development Notes

Some unit tests will use the AWS SDK, If you don’t already have a local AWS credentials file, create a dummy file called ~/.aws/credentials with the below contents:

[default]
aws_access_key_id = foobar
aws_secret_access_key = barfoo

Alternately, you can set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.