Initial prototype of aws-ecs-1 variant #946

samuelkarp · 2020-06-16T21:40:56Z

Issue number:
#815

Description of changes:
This pull request introduces a new variant of Bottlerocket called aws-ecs-1. The new variant includes the ECS agent and Docker to run containers.

The commits here represent the skeleton for packaging the ECS agent, defining the ECS model in the Bottlerocket API, rendering appropriate config, and making the new variant. I'd like to merge this (after review, of course) and then continue iterating on the variant in subsequent PRs.

Testing done:
Launched an AMI from this variant, observed that it connected to a non-default cluster (configured with the new settings.ecs.cluster setting) in ECS, and ran a basic task with minimal features (busybox:latest running sleep).

Set the new settings.ecs.instance-attributes setting through TOML-formatted user-data. Observed that the attributes are registered properly, along with the default bottlerocket.variant and bottlerocket.version attributes.

Toggled the new settings.ecs.privileged-containers-enabled setting through the Bottlerocket API. Observed that the config file was properly regenerated, the agent restarted, and the capabilities updated in the ECS API.

Validated that the pause container image is automatically loaded and able to run.

If you wish to test this variant, you can use this user-data:

[settings.ecs]
cluster = "bottlerocket"

[settings.ecs.instance-attributes]
hello = "world"

[settings.host-containers.admin]
enabled = true

Note: I have not yet updated the top-level README as the variant is not really ready for use yet. This means that the settings are currently undocumented.

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

packages/docker-engine/daemon.json

packages/ecs-init/ecs-init.spec

samuelkarp · 2020-06-16T22:49:57Z

(Note: CI failures are due to new package sources not being available in the cache yet.)

packages/ecs-init/0001-bottlerocket-remove-interaction-with-Nvidia-lib.patch

packages/ecs-init/0002-docker-do-not-bind-mount-missing-plugin-dirs.patch

jahkeup · 2020-06-16T23:06:17Z

packages/ecs-init/README.md

+  create any directory that does not exist, and `/usr/lib/docker/plugins` is
+  such a directory that does not exist.  However, Bottlerocket's filesystem is
+  mostly read-only, and ecs-init fails to create this directory.  For now,
+  bind-mounting is skipped when a directory does not exist, but this will break


I'm not familiar with how plugins are used, what functionality would this limit?

Docker plugins can be installed into those directories, see here. The agent scans these directories to discover unmanaged plugins. However, it does look like "managed" plugins (installed via the "docker plugin install" command) are typically installed into /run/docker/plugins which does work on Bottlerocket. It's probably reasonable for us to only support managed plugins on Bottlerocket, or unmanaged plugins built into Bottlerocket itself.

variants/aws-ecs-1/Cargo.toml

sources/api/ecs-settings-applier/src/main.rs

packages/ecs-agent/ecs.service

packages/ecs-agent/ecs-agent.spec

packages/ecs-agent/README.md

sources/models/src/lib.rs

packages/ecs-agent/ecs.service

packages/ecs-agent/Cargo.toml

packages/ecs-agent/ecs-agent.spec

samuelkarp · 2020-07-20T21:07:10Z

(GitHub won't let me reply on this)

@bcressey wrote:

Do we need to set net.ipv4.conf.all.route_localnet back to the default value on stop? Might be easiest to just remove it.

Probably not; this is one of the items in the README.md that I was looking for another way around. I don't really see a use-case for un-setting these options when ecs.service stops, but I also don't want to re-run the setup commands every time the service starts. My current idea (which I have not tried yet) is to move all these setup commands to a separate oneshot systemd service that the main ecs.service depends on.

webern

Impressive opus!

sources/api/ecs-settings-applier/src/main.rs

packages/ecs-agent/Cargo.toml

packages/ecs-agent/ecs-agent.spec

packages/os/os.spec

sources/api/ecs-settings-applier/Cargo.toml

sources/api/ecs-settings-applier/src/main.rs

sources/models/src/modeled_types/ecs.rs

packages/ecs-agent/0002-bottlerocket-default-filesystem-locations.patch

packages/ecs-agent/README.md

The systemd cgroup driver imposes additional restrictions on the name of cgroup parents, causing workflows that depend on non-conforming names to fail.

This macro can be used for conditions in variant-specific packages like `os`.

ecs-settings-applier is built conditionally for aws-ecs-1. Attempting to compile ecs-settings-applier for another variant will fail, as the applier depends on the definition of model::Model in the ECS variant.

The default docker0 bridge allows container networking to function in a familiar way to developers. The bridge is used by `docker run` when exposing ports and used by Amazon ECS.

See https://reproducible-builds.org/docs/archives/

samuelkarp requested a review from jahkeup June 16, 2020 21:40

bcressey reviewed Jun 16, 2020

View reviewed changes

packages/docker-engine/daemon.json Show resolved Hide resolved

packages/ecs-init/ecs-init.spec Outdated Show resolved Hide resolved

packages/ecs-init/ecs-init.spec Outdated Show resolved Hide resolved

jahkeup reviewed Jun 16, 2020

View reviewed changes

samuelkarp force-pushed the ecs branch from 65f3d08 to 90478b1 Compare June 23, 2020 19:20

jahkeup reviewed Jun 24, 2020

View reviewed changes

sources/api/ecs-settings-applier/src/main.rs Outdated Show resolved Hide resolved

sources/api/ecs-settings-applier/src/main.rs Show resolved Hide resolved

sources/api/ecs-settings-applier/src/main.rs Outdated Show resolved Hide resolved

samuelkarp force-pushed the ecs branch 3 times, most recently from 52656d8 to cdfe816 Compare July 1, 2020 20:03

samuelkarp force-pushed the ecs branch from cdfe816 to 851b625 Compare July 9, 2020 16:59

jahkeup reviewed Jul 10, 2020

View reviewed changes

packages/ecs-agent/ecs.service Outdated Show resolved Hide resolved

iliana reviewed Jul 15, 2020

View reviewed changes

packages/ecs-agent/ecs-agent.spec Outdated Show resolved Hide resolved

samuelkarp force-pushed the ecs branch from 851b625 to 141f707 Compare July 16, 2020 23:53

samuelkarp marked this pull request as ready for review July 17, 2020 00:01

samuelkarp requested review from iliana, bcressey, jahkeup and tjkirch July 17, 2020 00:01

samuelkarp force-pushed the ecs branch from 141f707 to 4763c0f Compare July 17, 2020 21:37

bcressey reviewed Jul 17, 2020

View reviewed changes

samuelkarp added 3 commits July 20, 2020 14:41

variants: Remove reference to top-level list

c0dd8d4

variants: create aws-ecs-1 variant

6295501

containerd: add config template for aws-ecs-1

062d3ad

samuelkarp force-pushed the ecs branch from 4763c0f to 854c2a8 Compare July 20, 2020 23:54

samuelkarp requested a review from bcressey July 21, 2020 00:07

webern approved these changes Jul 21, 2020

View reviewed changes

sources/api/ecs-settings-applier/src/main.rs Outdated Show resolved Hide resolved

samuelkarp force-pushed the ecs branch from 4d71d9e to 56cb12f Compare July 22, 2020 00:15

jahkeup reviewed Jul 22, 2020

View reviewed changes

sources/api/ecs-settings-applier/src/main.rs Outdated Show resolved Hide resolved

etungsten reviewed Jul 22, 2020

View reviewed changes

fenxiong reviewed Jul 25, 2020

View reviewed changes

packages/ecs-agent/0002-bottlerocket-default-filesystem-locations.patch Outdated Show resolved Hide resolved

packages/ecs-agent/README.md Show resolved Hide resolved

fenxiong approved these changes Jul 25, 2020

View reviewed changes

samuelkarp and others added 24 commits July 24, 2020 18:44

ecs-agent: new package

d9d430f

docker-engine: use cgroupfs cgroup driver

a7a7971

The systemd cgroup driver imposes additional restrictions on the name of cgroup parents, causing workflows that depend on non-conforming names to fail.

aws-ecs-1: add ecs-agent package

169b4d0

packages: define %{_cross_variant} macro

75d49e9

This macro can be used for conditions in variant-specific packages like `os`.

ecs-settings-applier: initial implementation

0e06a20

os: build ecs-settings-applier for aws-ecs-1

75d4f04

ecs-settings-applier is built conditionally for aws-ecs-1. Attempting to compile ecs-settings-applier for another variant will fail, as the applier depends on the definition of model::Model in the ECS variant.

aws-ecs-1: use ecs-settings-applier

c9b34a1

docker-engine: allow Docker to use default bridge

8727e78

The default docker0 bridge allows container networking to function in a familiar way to developers. The bridge is used by `docker run` when exposing ports and used by Amazon ECS.

ecs-settings-applier: use snafu for errors

44b6499

ecs-settings-applier: add --socket-path arg

82d497d

ecs-settings-applier: add default attributes

2a8ea51

models: add instance attributes for aws-ecs-1

cfcd781

ecs-settings-applier: render attributes

1d6ef49

aws-ecs-1: add setting for privileged containers

a0ad8bd

ecs-settings-applier: render privileged containers

b69acb5

aws-ecs-1: default disable privileged containers

9d7b097

ecs-agent: build pause image archive reproducibly

061f2fd

See https://reproducible-builds.org/docs/archives/

.github: add aws-ecs-1 variant to Build workflow

4587c69

ecs-settings-applier: destructure ecs settings

0debe9f

ecs-agent: use systemd-sysctl and sysctl.d

3402fa7

Unpack expected Options

950a0ff

ecs-settings-applier: add README with cargo-readme

b95c80f

ecs-settings-applier: simplify ECSConfig cluster

50a427b

ecs-settings-applier: cargo fmt

8025807

samuelkarp force-pushed the ecs branch from 0eacfe7 to 8025807 Compare July 25, 2020 01:45

fenxiong approved these changes Jul 27, 2020

View reviewed changes

samuelkarp merged commit 988c0de into bottlerocket-os:develop Jul 27, 2020

samuelkarp deleted the ecs branch July 27, 2020 22:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial prototype of aws-ecs-1 variant #946

Initial prototype of aws-ecs-1 variant #946

samuelkarp commented Jun 16, 2020 •

edited

Loading

samuelkarp commented Jun 16, 2020

jahkeup Jun 16, 2020

samuelkarp Jun 17, 2020

samuelkarp commented Jul 20, 2020

webern left a comment

Initial prototype of aws-ecs-1 variant #946

Initial prototype of aws-ecs-1 variant #946

Conversation

samuelkarp commented Jun 16, 2020 • edited Loading

samuelkarp commented Jun 16, 2020

jahkeup Jun 16, 2020

Choose a reason for hiding this comment

samuelkarp Jun 17, 2020

Choose a reason for hiding this comment

samuelkarp commented Jul 20, 2020

webern left a comment

Choose a reason for hiding this comment

samuelkarp commented Jun 16, 2020 •

edited

Loading