Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for S3 SSE with KMS #29

Merged
merged 1 commit into from
Aug 14, 2017
Merged

Add support for S3 SSE with KMS #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/config-definition.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ The configurable parameters are as follows:
| `disableSSL` | bool | `false` | Set this to `true` if you are using Minio (or another local, S3-compatible storage service) and your deployment is not secured. |
| `s3ForcePathStyle` | bool | `false` | Set this to `true` if you are using a local storage service like Minio. |
| `s3Url` | string | Required field for non-AWS-hosted storage| *Example*: http://minio:9000<br><br>You can specify the AWS S3 URL here for explicitness, but Ark can already generate it from `region`, `availabilityZone`, and `bucket`. This field is primarily for local sotrage services like Minio.|
| `kmsKeyID` | string | Empty | *Example*: "502b409c-4da1-419f-a16e-eif453b3i49f"<br><br>Specify an [AWS KMS key][12] id to enable encryption of the backups stored in S3. Only works with AWS S3 and may require explicitly granting key usage rights.|

### GCP
| Key | Type | Default | Meaning |
Expand All @@ -95,3 +96,5 @@ The configurable parameters are as follows:
[9]: #main-config-parameters
[10]: #overview
[11]: #example
[12]: http://docs.aws.amazon.com/kms/latest/developerguide/overview.html

1 change: 1 addition & 0 deletions pkg/apis/ark/v1/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ type AWSConfig struct {
DisableSSL bool `json:"disableSSL"`
S3ForcePathStyle bool `json:"s3ForcePathStyle"`
S3Url string `json:"s3Url"`
KMSKeyID string `json:"kmsKeyId"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got tripped up on this being kmsKeyId and not kmsKeyID. It looks like we're inconsistent in other keys (s3Url rather than s3URL but disableSSL, not disableSsl). @ncdc thoughts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible some of the discrepancies are from me manually generating some tags vs using vim-go's :GoAddTags - can't say for sure. Consistency would be nice... But not something we need to change right now. Especially since it breaks backwards compat.

}

// GCPConfig is configuration information for connecting to GCP.
Expand Down
10 changes: 9 additions & 1 deletion pkg/cloudprovider/aws/object_storage_adapter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ package aws
import (
"io"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/s3"

"github.com/heptio/ark/pkg/cloudprovider"
Expand All @@ -27,7 +28,8 @@ import (
var _ cloudprovider.ObjectStorageAdapter = &objectStorageAdapter{}

type objectStorageAdapter struct {
s3 *s3.S3
s3 *s3.S3
kmsKeyID string
}

func (op *objectStorageAdapter) PutObject(bucket string, key string, body io.ReadSeeker) error {
Expand All @@ -37,6 +39,12 @@ func (op *objectStorageAdapter) PutObject(bucket string, key string, body io.Rea
Body: body,
}

// if kmsKeyID is not empty, enable "aws:kms" encryption
if op.kmsKeyID != "" {
req.ServerSideEncryption = aws.String("aws:kms")
req.SSEKMSKeyId = &op.kmsKeyID
}

_, err := op.s3.PutObject(req)

return err
Expand Down
5 changes: 3 additions & 2 deletions pkg/cloudprovider/aws/storage_adapter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ type storageAdapter struct {

var _ cloudprovider.StorageAdapter = &storageAdapter{}

func NewStorageAdapter(config *aws.Config, availabilityZone string) (cloudprovider.StorageAdapter, error) {
func NewStorageAdapter(config *aws.Config, availabilityZone string, kmsKeyID string) (cloudprovider.StorageAdapter, error) {
sess, err := session.NewSession(config)
if err != nil {
return nil, err
Expand All @@ -48,7 +48,8 @@ func NewStorageAdapter(config *aws.Config, availabilityZone string) (cloudprovid
az: availabilityZone,
},
objectStorage: &objectStorageAdapter{
s3: s3.New(sess),
s3: s3.New(sess),
kmsKeyID: kmsKeyID,
},
}, nil
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/cmd/server/server.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -324,7 +324,7 @@ func getAWSCloudProvider(cloudConfig api.CloudProviderConfig) (cloudprovider.Sto
)
}

return arkaws.NewStorageAdapter(awsConfig, cloudConfig.AWS.AvailabilityZone)
return arkaws.NewStorageAdapter(awsConfig, cloudConfig.AWS.AvailabilityZone, cloudConfig.AWS.KMSKeyID)
}

func getGCPCloudProvider(cloudConfig api.CloudProviderConfig) (cloudprovider.StorageAdapter, error) {
Expand Down