Notice: aws_vpc_endpoint_service: Error: multiple VPC Endpoint Services matched

[breathingdust]

Problem

For those users who are encountering the following error when using the aws_vpc_endpoint_service data source:

 Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

• AWS have just released a new feature in S3 (PrivateLink) which means that multiple results are now being returned when searching for the S3 endpoint service.
• Singular data sources in the Terraform AWS Provider (like aws_vpc_endpoint_service) return an error if multiple results are returned.

Configuration changes required to resolve the issue

Add a filter block to select a service type, e.g.

data "aws_vpc_endpoint_service" "s3" {
  service = "s3"

  filter {
    name   = "service-type"
    values = ["Gateway"]
  }
}

For provider versions v3.10.0 and up, it is also possible to use the service_type argument for simplifying the configuration:

data "aws_vpc_endpoint_service" "s3" {
  service      = "s3"
  service_type = "Gateway"
}

Please note that if you are using a Terraform module that relies on this datasource, the module itself will need to be updated. Terraform modules are not maintained by HashiCorp, so you will need to reach out to the modules maintainers to make that configuration update.

References

• data-source/aws_vpc_endpoint_service: Accept service_type as argument #15467

[circa10a]

if you are using a provider version prior to v3 this won't work for aws_vpc_endpoint_service data sources since multiple items will be returned due to the new vpc service endpoint being the same name and service type filtering was added in v3. Current supported filters via the API are only the name and tags.

data "aws_region" "current" {}

data "aws_vpc_endpoint_service" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  service = "s3"
  filter {
    name   = "service-name"
    values = ["com.amazonaws.${data.aws_region.current.name}.s3"]
  }
}

Result:

Error: multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service

When running aws ec2 describe-vpc-endpoint-services --region us-east-1 : here's output of what the service endpoints' config look like(no tags):

 {
            "ServiceName": "com.amazonaws.us-east-1.s3",
            "ServiceId": "vpce-svc-*****",
            "ServiceType": [
                {
                    "ServiceType": "Gateway"
                }
            ],
            "AvailabilityZones": [
                "us-east-1a",
                "us-east-1b",
                "us-east-1c",
                "us-east-1d",
                "us-east-1e",
                "us-east-1f"
            ],
            "Owner": "amazon",
            "BaseEndpointDnsNames": [
                "s3.us-east-1.amazonaws.com"
            ],
            "VpcEndpointPolicySupported": true,
            "AcceptanceRequired": false,
            "ManagesVpcEndpoints": false,
            "Tags": []
        },
        {
            "ServiceName": "com.amazonaws.us-east-1.s3",
            "ServiceId": "vpce-svc-****",
            "ServiceType": [
                {
                    "ServiceType": "Interface"
                }
            ],
            "AvailabilityZones": [
                "us-east-1a",
                "us-east-1b",
                "us-east-1c",
                "us-east-1d",
                "us-east-1e",
                "us-east-1f"
            ],
            "Owner": "amazon",
            "BaseEndpointDnsNames": [
                "s3.us-east-1.vpce.amazonaws.com"
            ],
            "VpcEndpointPolicySupported": true,
            "AcceptanceRequired": false,
            "ManagesVpcEndpoints": false,
            "Tags": []
        },

[circa10a]

❗ See better solution below

Workaround for now is to tag the service endpoint:

aws ec2 create-tags --resources vpce-svc-***** --tag Key=type,Value=gateway --region us-east-1

Then in terraform

data "aws_vpc_endpoint_service" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  service = "s3"
  filter {
    name   = "tag:type"
    values = ["gateway"]
  }
}

[circa10a]

And here's a crufty script to tag the existing s3 gateway service endpoints in multiple regions, should you need to

 #!/usr/bin/env bash
 
 export AWS_PAGER=""
 REGIONS=("us-east-1" "us-east-2" "us-west-1" "us-west-2" "ap-northeast-1" "ap-northeast-2" "ap-southeast-1" "ap-southeast-2" "ca-central-1" "eu-central-1" "eu-west-1" "eu-west-2" "eu-west-3" "sa-east-1")

 for REGION in "${REGIONS[@]}";do
   echo "Tagging region: ${REGION}"
   S3_GATEWAY_ENDPOINT_ID=$(aws ec2 describe-vpc-endpoint-services --region ${REGION} --output json | grep -C 5 s3 | grep -B 3 Gateway | grep -oE "vpce.*" | sed 's/",//g')
   echo "S3 Gateway ID: ${S3_GATEWAY_ENDPOINT_ID}"
   aws ec2 create-tags --resources "$S3_GATEWAY_ENDPOINT_ID" --tag Key=type,Value=gateway --region "$REGION"
 done

[endlesslight0]

The configuration fix is to specify the service_type argument, which was added in Terraform AWS Provider version 3.10.0:

data "aws_vpc_endpoint_service" "s3" {
  service      = "s3"
  service_type = "Gateway"
}

Using AWS provider version 3.24.1 and with the solution above still getting the same error.

---

Update:
upgrading terraform vpc module to latest version has helped solve the issue.

[kelvinmatheus]

Workaround for now is to tag the service endpoint:

aws ec2 create-tags --resources vpce-svc-***** --tag Key=type,Value=gateway --region us-east-1

Then in terraform

data "aws_vpc_endpoint_service" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  service = "s3"
  filter {
    name   = "tag:type"
    values = ["gateway"]
  }
}

Thanks a lot, it worked here with some changes. We have just created the filter and changed the count assignment a little bit.

count = "${var.enable_s3_endpoint ? 1 : 0}"

[karkrish]

Guys, I got another workaround for this issue. I removed the data section and integrated with endpoint resource.

resource "aws_vpc_endpoint" "s3" {
  count = "${var.create_vpc && var.enable_s3_endpoint ? 1 : 0}"
  vpc_id       = "XXXX"
  service_name = "com.amazonaws.${var.aws_region}.s3"
  vpc_endpoint_type = "Gateway"
  tags {
    Name = "XXXXX"
  }
}

[circa10a]

Guys, I got another workaround for this issue. I removed the data section and integrated with endpoint resource.

resource "aws_vpc_endpoint" "s3" {
count = "${var.create_vpc && var.enable_s3_endpoint ? 1 : 0}"
vpc_id = "XXXX"
service_name = "com.amazonaws.${var.aws_region}.s3"
vpc_endpoint_type = "Gateway"
tags {
Name = "XXXXX"
}
}

Fantastic 🎉 , confirmed this is supported with provider 2.70.0

Here's how we implemented

data "aws_region" "current" {}

resource "aws_vpc_endpoint" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  vpc_id            = local.vpc_id
  service_name      = "com.amazonaws.${data.aws_region.current.name}.s3"
  vpc_endpoint_type = var.s3_vpc_endpoint_type # default = "Gateway"
  tags              = local.vpce_tags
}

[fazalmasood]

Error: "service_type": this field cannot be set

on .terraform/modules/iq_networking/main.tf line 323, in data "aws_vpc_endpoint_service" "s3":
323: data "aws_vpc_endpoint_service" "s3" {
[script-executor] Script completed with exit code: 1.

Other details

• Checking for available provider plugins...
• Downloading plugin for provider "tls" (hashicorp/tls) 3.0.0...
• Downloading plugin for provider "aws" (hashicorp/aws) 2.70.0...
• Downloading plugin for provider "null" (hashicorp/null) 3.0.0...

[fazalmasood]

Could some one help me here to solve this issue

[circa10a]

@fazalmasood This is because you are using provider version 2.70.0. I mentioned the fix for that provider version just above. You essentially need to get rid of the data source, but if you can't I also listed an alternative method further up in the issue.

Guys, I got another workaround for this issue. I removed the data section and integrated with endpoint resource.
resource "aws_vpc_endpoint" "s3" {
count = "${var.create_vpc && var.enable_s3_endpoint ? 1 : 0}"
vpc_id = "XXXX"
service_name = "com.amazonaws.${var.aws_region}.s3"
vpc_endpoint_type = "Gateway"
tags {
Name = "XXXXX"
}
}

Fantastic 🎉 , confirmed this is supported with provider 2.70.0

Here's how we implemented

data "aws_region" "current" {}

resource "aws_vpc_endpoint" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  vpc_id            = local.vpc_id
  service_name      = "com.amazonaws.${data.aws_region.current.name}.s3"
  vpc_endpoint_type = var.s3_vpc_endpoint_type # default = "Gateway"
  tags              = local.vpce_tags
}

[kjsingh]

Verified on Terraform version 0.11:

resource 
"aws_vpc_endpoint" "s3" 
{

vpc_id = 
"${module.vpc.vpc_id}"

service_name = 
"com.amazonaws.${local.region}.s3"

vpc_endpoint_type = 
"Gateway"

}

[rust84]

We are using the workaround provided by @circa10a and this has resolved the issue for us. Hopefully terraform-aws-modules/terraform-aws-vpc#587 get's accepted soon.

[BradH19]

Verified on Terraform version 0.11:

resource 
"aws_vpc_endpoint" "s3" 
{

vpc_id = 
"${module.vpc.vpc_id}"

service_name = 
"com.amazonaws.${local.region}.s3"

vpc_endpoint_type = 
"Gateway"

}

What AWS Provider version are you using? I'm still getting the same error (shown below), even when using the region.
"multiple VPC Endpoint Services matched; use additional constraints to reduce matches to a single VPC Endpoint Service"

[kjsingh]

have you removed the data block?

data "aws_vpc_endpoint_service" "s3" {...}

[BradH19]

@kjsingh That may have been it. I've just removed that data block and ran plan with no errors. Still need to do some other testing but this is looking much better than it did a few hours ago.

[kumartushar]

Verified on Terraform version 0.11:

resource 
"aws_vpc_endpoint" "s3" 
{

vpc_id = 
"${module.vpc.vpc_id}"

service_name = 
"com.amazonaws.${local.region}.s3"

vpc_endpoint_type = 
"Gateway"

}

Thanks this worked 👍

[circa10a]

Update: AWS added a new filter to the DescribeVPCEndpointServices API which will now allow for this:

data "aws_vpc_endpoint_service" "s3" {
  count = var.create_vpc && var.enable_s3_endpoint ? 1 : 0

  service = "s3"
  filter {
    name   = "service-type"
    values = ["Gateway"]
  }
}

This is new API functionality that will work with any client, independent of terraform/provider versions

[circa10a]

FYI that v2.74.0 of the terraform-aws-vpc module was updated the be compatible with v2 and v3 of the terraform AWS provider.

[MysticalMount]

Hi all, we fixed this in the interim with a custom bash script (https://github.com/MysticalMount/aws_vpc_endpoint_service) - forgive me for lack of knowledge but it looks like the v2.74.0 vpc module mentioned above might be a more appropriate fix - will this automatically install along with the AWS provider in version 11 of Terraform now or are there some other steps - does this module form part of the AWS provider itself?

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.