Glue

AWS Glue Well-Architected Custom Lens

• About
• Prepare the json file
• Create custom lens on Well-Architected Tool
• Intended Audience
• Contributing

About

AWS Glue Well-Architected Custom Lens specifies best practices in Operational Excellence, Security, Performance Efficiency and Cost Optimization pillars. The best practices are designed for Glue workload.

Prepare the json file

Prepare your custom lens Well-Architected template json file or just download the provided recommended glue custom lens file.

{
   "schemaVersion":"2021-11-01",
   "name":"Glue Best Practice Lens",
   "description":"Best practices for configuring Glue",
   "pillars":[
      {
         "id":"PERF",
         "name":"Performance Efficiency",
         "questions":[
            {
               "id":"PERF1",
               "title":"Do konw how file formats, file size, file layout and compression effect your job performance?",
               "description":"To reduce the amount of data loaded into your job when reading from Amazon S3, you need to consider FileSize, Compression, FileFormat and FileLayout (Partitions) for your dataset.",
               "choices":[
                  {
                     "id":"PERF1_1",
                     "title":"Choice the suitable file format for you Glue ETL Job",
                     "helpfulResource":{
                        "displayText":"Have you use a columnar format. Apache Parquet and Apache ORC are popular columnar data formats?\n\nWhen using columnar formats, you can skip blocks of data that correspond to columns you do not plan to use.",
                        "url":"https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/"
                     },
                     "improvementPlan":{
                        "displayText":"Use input/output parquet, orc dataformat",
                        "url":"https://aws.amazon.com/blogs/big-data/top-10-performance-tuning-tips-for-amazon-athena/"
                     }
                  }
               ]
            }
         ]
      }
   ]
}

Create custom lens on Well-Architected Tool

After you prepare your custom lens Well-Architected template json file, you can navigate to the AWS Well-Architected Tool on the AWS console and create custom lens.

Upload the your custom lens Well-Architected template json template, and submit it.

And then you will find your Well-Architected custom lens in draft version.

Publish your draft version and provide a version tag.

After your Well-Architected custom lens is published, you can define a workload in your AWS Well-Architected console.

Input your workload information, and select the Well-Architected custom lens you just published.

Choose Continue reviewing for the just published Glue Well-Architected custom lens.

Now you can review your Glue workload with Glue Well-Architected Custom Lens.

Intended Audience

CTO, Technical Leader or technical owner for an AWS Glue workload

Contributing

• Noritaka Sekiyama, Principal Big Data Architect, AWS
• Ray Wang, Solutions Architect, AWS
• Hsu, Chia-Wei Cloud Support Engineer (BigData), Glue SME, AWS