Cannot create S3 VPC Endpoint with private DNS enabled.

[shadow-wheel]

Description

When trying to create an S3 endpoint with private_dns_enabled = true, an error is returned when using the vpc-endpoints module. I've verified the issue with aws provider version 5.16.2 and this vpc module v5.1.2.

The error message is

│ Error: creating EC2 VPC Endpoint (com.amazonaws.us-west-2.s3): InvalidParameter: To set PrivateDnsOnlyForInboundResolverEndpoint to true, the VPC vpc-xxxxxxx must have a Gateway endpoint for the service.
│       status code: 400, request id: cc137867-946f-4e06-ada7-7f5a153440d4
│ 
│   with module.application_vpc.module.vpc_endpoints.aws_vpc_endpoint.this["s3"],
│   on .terraform/modules/application_vpc.vpc_endpoints/modules/vpc-endpoints/main.tf line 23, in resource "aws_vpc_endpoint" "this":
│   23: resource "aws_vpc_endpoint" "this" {

This is referencing aws provider bug mentioned in hashicorp/terraform-provider-aws#32407 and hashicorp/terraform-provider-aws#31117

Per the docs https://aws.amazon.com/blogs/storage/introducing-private-dns-support-for-amazon-s3-with-aws-privatelink/ it seems that Scenario 2 (all traffic routed via private endpoint) is not possible. The default configuration expects to have both a gateway endpoint to S3 (for VPC traffic) and an Interface endpoint (to route on-prem traffic).

We cannot enable a single interface endpoint because the dns_options. private_dns_only_for_inbound_resolver_endpoint seems to be getting set to true if private_dns_enabled is set to true and the vpc-endpoints module has no provision to override this.

The workaround is to create both a gateway and an interface endpoint to satisfy requirements of scenario 3 in the docs or to create a standalone aws_vpc_endpoint resource and not use the vpc-endpoints module for this at all. We have a need to have all traffic to S3 be private so the first workaround of creating both types of endpoints is not an option.

• ✋ I have searched the open/closed issues and my issue is not listed.

Versions

• Module version [Required]: 5.1.2

• Terraform version:
  1.4.6

• Provider version(s):

• provider registry.terraform.io/hashicorp/aws v5.16.2

Reproduction Code [Required]

Example:

module "vpc_endpoints" {
  source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"
  version = "v5.1.2"

  vpc_id             = module.vpc.vpc_id
  security_group_ids = [module.vpc_endpoints_sg.security_group_id]

  endpoints = {
    s3 = {
      service             = "s3"
      tags                = { Name = "s3-vpc-endpoint" }
      service_type        = "Interface"
      private_dns_enabled = true
    }
  }

  tags = var.default_tags

}

Steps to reproduce the behavior:

see above.

Expected behavior

Only specify an interface endpoint for S3 with private_dns_enabled: true and private_dns_only_for_inbound_resolver_endpoint: false to allow for all traffic to S3 be private.

Actual behavior

Both referenced parameters are set to true and there is not option to explicitly set private_dns_only_for_inbound_resolver_endpoint to false within the module.

Terminal Output Screenshot(s)

n/a

Additional context

The module should allow for setting options within the dns_options block as noted in the resource documentation: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint#dns_options

[bryantbiggs]

Apologies if I'm missing something, but I don't see this as a module issue. Is there a configuration combination that we do not support (I don't think so)? Otherwise, it's down to the configuration provided by users and whether or not the AWS API accepts that configuration

[shadow-wheel]

Apologies if I'm missing something, but I don't see this as a module issue. Is there a configuration combination that we do not support (I don't think so)? Otherwise, it's down to the configuration provided by users and whether or not the AWS API accepts that configuration

Well, it is a module issue in that here in the module, when declaring the aws_vpc_endpoint resource, there is no provision to pass in dns_options. This is relating to a relatively new feature introduced by AWS in June 2023 so I guess from my perspective I'm waiting for the module to catch up.

I'm sorry if my description above was confusing, I was just trying to provide as much information as possible.

[bryantbiggs]

ah - now that I can understand, thank you for that! we can support this

[alanwu4321]

@bryantbiggs Do we have a workaround? If not I can try to contribute

But seems like upstream has a bug before we can implement it

hashicorp/terraform-provider-aws#33689

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[antonbabenko]

This issue has been resolved in version 5.5.0 🎉

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.