Bluegrass

A sample MLOps system of LeapMind Blueoil x AWS Services (AWS IoT Greengrass, Amazon SageMaker)

AWS Partner Network (APN) Blog

The following APN blog describes how to train models and run inference on Intel FPGA Edge Devices by using this repository.

• Using Fewer Resources to Run Deep Learning Inference on Intel FPGA Edge Devices

System Outline

Setup AWS environment and devices

Prerequisites

• AWS account
• Terasic DE10-Nano Kit x 2

Requirements

You need following packages to setup components.

AWS CLI

Please set up the AWS Command Line Interface (AWS CLI), check official User Guide. You can choose the region where Amazon SageMaker and AWS IoT Greengrass are supported (see Region Table).

Ansible

You can install with pip command.

$ pip3 install -r requirements.txt

Prepare your Amazon S3 bucket and Roles for deployment

Decide your Amazon S3 bucket name. Bucket names must be unique across all existing bucket names in Amazon S3. Please see Bucket Restrictions and Limitations.

$ export BLUEGRASS_S3_BUCKET_NAME=[your S3 bucket name]
$ aws cloudformation create-stack --stack-name BluegrassS3 --capabilities CAPABILITY_NAMED_IAM --template-body file://$(pwd)/deploy/s3.yaml --parameters ParameterKey="S3BucketName",ParameterValue="${BLUEGRASS_S3_BUCKET_NAME}"

Please make sure your bucket is created. Check status from your cloudformation console.

Blueoil x Amazon SageMaker

Create an Amazon SageMaker Notebook instance for training with Blueoil

$ aws cloudformation create-stack --stack-name BlueoilSagemaker --template-body file://$(pwd)/deploy/sagemaker.yaml

Run training on Amazon SageMaker

Open Jupyter/JupyterLab on the Amazon SageMaker console. Find and run the notebook in the bluegrass/blueoil_sagemaker directory, e.g., blueoil_cifar10_example.ipynb or blueoil_openimages_example.ipynb .

Blueoil x AWS IoT Greengrass

Create certificate files for AWS IoT Greengrass

Run following script.

$ deploy/create_cert.sh
...
Certificate files are created in certs/xxxxxx...xxxxxx
certificateArn is "arn:aws:iot:xxxxxx:xxxxxx:cert/xxxxxx...xxxxxx"

Please save your certificateArn, this will be used when creating Greengrass components in your AWS account.

In this sample, you need to run this command twice to create certicicates for each DE10-Nano device.

Create Greengrass components for DE10-Nano

Run commands below, where MODEL_S3_URI will be set as a form of s3://sagemaker-xxxxxx/blueoil-sagemaker-2020-XX-XX-XX-XX-XX-XXZ/output/converted/output.tar.gz.

$ export MODEL_S3_URI=[your S3 uri]
$ export CERT_ARN1=[your certificateArn1]
$ export CERT_ARN2=[your certificateArn2]
$ aws cloudformation create-stack --stack-name Bluegrass --capabilities CAPABILITY_NAMED_IAM --template-body "$(aws cloudformation package --template-file deploy/greengrass.yaml --s3-bucket ${BLUEGRASS_S3_BUCKET_NAME})" \
--parameters \
ParameterKey="ModelS3Uri",ParameterValue="${MODEL_S3_URI}" \
ParameterKey="S3BucketName",ParameterValue="${BLUEGRASS_S3_BUCKET_NAME}" \
ParameterKey="Core01CertificateArn",ParameterValue="${CERT_ARN1}" \
ParameterKey="Core02CertificateArn",ParameterValue="${CERT_ARN2}"

Setup DE10-Nano

Download the greengrass certified SD card image from here

Write that image to your SD card and boot it. Setup network and ssh-keys to enable to access your de10nano via network without password.

Run ansible to install greengrass and setup for blueoil-inference.

$ ansible-playbook -i [IP address of Core1],[IP address of Core2] ansible/playbook.yml

Upload certificate and config files to DE10-Nano

Get your cert_id from ${CERT_ARN1}, ${CERT_ARN2}.

$ export CERT_ID1=$(echo ${CERT_ARN1} | sed -e 's/.*\///g')
$ export CERT_ID2=$(echo ${CERT_ARN2} | sed -e 's/.*\///g')

Check your thing_arn of your Core device. Find thing_arn for your core in the AWS IoT Greengrass console under Cores.

$ export THING_ARN1=[thing arn of your Core1]
$ export THING_ARN2=[thing arn of your Core2]

Set your IOT_HOST and GG_HOST.

$ export IOT_HOST=$(aws iot describe-endpoint --endpoint-type iot:Data-ATS --query 'endpointAddress' --output=text)
$ export GG_HOST=$(echo ${IOT_HOST} | sed -e 's/.*-ats/greengrass-ats/g')

Deploy cert and config files to DE10-Nano.

# For Device of Core1
$ ansible-playbook -i [IP address of Core1], ansible/playbook_certs_deploy.yml --extra-vars "cert_id=${CERT_ID1} thing_arn=${THING_ARN1} iot_host=${IOT_HOST} gg_host=${GG_HOST}"
# For Device of Core2
$ ansible-playbook -i [IP address of Core2], ansible/playbook_certs_deploy.yml --extra-vars "cert_id=${CERT_ID2} thing_arn=${THING_ARN2} iot_host=${IOT_HOST} gg_host=${GG_HOST}"

Associate serivce role to your account

Before first deployment, you need to associate service role to your account for greengrass deployment. This needs only once for your account.

$ ./deploy/associate_service_role.sh
{
    "AssociatedAt": "2020-XX-XXTXX:XX:XXZ"
}

Initial deployment

Manualy deployment from the AWS IoT console

After completing the above installations, you need to deploy a model and an application to devices manually.

Access to the AWS IoT console.

Select the group and click [Action] -> [Deploy].

The following screen is shown at the first deployment to the group. Select [Automatic detection].

Confirm that the status is changed as [In Progress]->[Successfully Completed] as shown below.

Check inference results

You can see the video and inference results by accessing to http://[device's IP address]:8080

Update components

Update AWS Lambda function

After updating Lambda function, you can deploy it.

$ cd deploy/lambda_function
$ ./deploy_lambda.sh

Update models

Please use the AWS IoT Greengrass console to change S3 path of machine learning resources.