Automatically optimize your images & videos hosted on AWS S3 when an S3 event is detected
This project is inspired by image-optimizer & Lambda ECS Worker Pattern
A tag is also applied to the optimized file so this file will be ignored in subsequent event
- S3
- S3 is configured to trigger a specific lambda on objectCreate event
- Lambda
- catch the S3 Event
- push the event to a SQS queue
- EC2
- constantly listens to messages from SQS queue
- when a message arrives :
- get the S3 Event from the message
- get information about S3 object (content-type)
- download the S3 object
- executes optimization chain on object (one or more command)
- upload the optimized object to S3
In AWS dashboard, go to cloudformation & create a new stack from the template file cloudformation.yml
In the parameters, change the bucket name & the subnet value (a private subnet is fine)
When the stack is up, go to the created S3 bucket & upload some images. After some moment, you will notice they will be automatically optimized
You can check that the new optimized image has the tag optimizer_ignore
:
In CloudWatch you can get the logs of your Lambda & your docker container running the optimization program
The optimization configuration is located in optimizer.json file. In the actual version, the JSON file need to be at the same level as the media-optimizer executable
Currently using all the optimizers from image-optimizer project + ffmpeg for mp4 video :
- image/png
pngquant --output [output_file] -f [intput_file]
optipng -out [output_file] -i0 -o2 -clobber [intput_file]
- image/jpeg
jpegoptim -d [output_directory] -m85 --strip-all --all-progressive [intput_file]
- image/svg+xml
svgo -o [output_file] --disable={cleanupIDs,removeViewBox} [intput_file]
- image/gif
gifsicle -o [output_file] -b -O3 [intput_file]
- image/webp
cwebp -o [output_file] -m 6 -pass 10 -mt -q 80 [intput_file]
- video/mp4
ffmpeg -i [input_file] -vcodec libx264 -crf 24 -y [output_file]
You can add more format & more commands to the config file optimizer.json
{
"optimizers": [{
"contentType": "image/png",
"exec": [{
"binary": "pngquant",
"outputFile": "--output",
"params": ["-f"]
},{
"binary":"optipng",
"outputFile": "-out",
"params": ["-i0","-o2", "-clobber"]
}]
.....
}
binary
is the command to executeparams
are the command parametersoutputFile
is the parameter for the output fileoutputDirectory
is the parameter for the output directory ifoutputFile
is not specifiedinputFile
is the parameter for the input file if neitheroutputFile
noroutputDirectory
are specified. In caseinputFile
is specified the outputFile value will be appended to the parameters (at the end)
Those variables configure S3 Object configuration & metadata :
name | default value |
---|---|
OBJECT_ACL | private |
OBJECT_CACHE_CONTROL | max-age=15552000 |
OBJECT_STORAGE_CLASS | STANDARD |
OBJECT_SERVER_SIDE_ENCRYPTION | none |
Note that if AWS::Region
is not specified in your ~/.aws
, the variable AWS_REGION
should be set (see cloudformation.yml)
make install
make run
docker build . -t media-optimizer
docker run -v $HOME/.aws:/root/.aws -it media-optimizer
docker pull bertrandmartel/media-optimizer:latest
docker run -v $HOME/.aws:/root/.aws -it bertrandmartel/media-optimizer:latest