Fixing DNS Resolution Issues in Docker Containers with a Custom `resolv.conf`

When running Docker containers, you might encounter DNS resolution issues due to the default search domain settings inherited from the host. Specifically, the search ec2.internal directive can interfere with proper DNS resolution within containers. This document describes a solution to address this issue by using a custom resolv.conf file.

Solution Overview

The solution involves creating a custom resolv.conf file with specific DNS settings and mounting this file into each Docker container. This ensures consistent and correct DNS resolution without unwanted search domains.

Steps to Implement the Solution

1. Create a Custom resolv.conf File: First, find the current nameserver & options values by looking into one of the containers.

   docker compose exec alpaca bash -lc "cat /etc/resolv.conf" | grep nameserver
   docker compose exec alpaca bash -lc "cat /etc/resolv.conf" | grep options

   Create a custom resolv.conf file with the desired DNS settings. The file will use Docker's internal DNS server at 127.0.0.11 and remove any search domains. You might also prefer to set the search domain to the nameserver's IP instead of period.

   sudo bash -c 'echo -e "nameserver 127.0.0.11\nsearch .\noptions edns0 trust-ad ndots:0" > /etc/docker/custom-resolv.conf'
   
   # OR if you're looking to automate.
   sudo bash -c 'echo -e "$(docker compose exec alpaca bash -lc "cat /etc/resolv.conf" | grep nameserver)\nsearch .\n$(docker compose exec alpaca bash -lc "cat /etc/resolv.conf" | grep options)" > /etc/docker/custom-resolv.conf'

   The content of /etc/docker/custom-resolv.conf should be:

   nameserver 127.0.0.11
   search .
   options edns0 trust-ad ndots:0

   • nameserver 127.0.0.11: Uses Docker's internal DNS server.
   • search .: Sets the search domain to the root domain, effectively removing any search domain.
   • options edns0 trust-ad ndots:0: Additional DNS options to enhance DNS resolution and security.
   • Leaving search domain blank result in defaulting to the host's search domain upon running docker compose up

2. Update Docker Compose Configuration:

   Modify your docker-compose.SERVICE.yml file to mount the custom resolv.conf file into each service. This ensures that the custom DNS settings are used by all containers.

   Example docker-compose.yml:

   services:
      alpaca:
         restart: ${RESTART_POLICY:-unless-stopped}
         image: ${REPOSITORY:-islandora}/alpaca:${TAG:-latest}
         volumes:
            - /etc/docker/custom-resolv.conf:/etc/resolv.conf

   Add the volumes section to each service that requires the custom DNS settings. This mounts the custom resolv.conf from the host into the container.

3. Deploy the Updated Docker Compose Configuration:

   Apply the changes by re-deploying your Docker Compose setup. This can be done using the following command:

   docker compose down
   docker compose up -d

   This command will recreate the containers with the new configuration, ensuring they use the custom resolv.conf file.

Verification

1. Inspect resolv.conf in Containers:

   After deploying the updated configuration, inspect the /etc/resolv.conf file inside a running container to verify that it uses the custom settings.

   docker compose exec alpaca bash -lc "cat /etc/resolv.conf"

   Ensure that the content matches the custom resolv.conf file you created.

2. Test DNS Resolution:

   Test DNS resolution within the container to ensure that it works as expected without the interference of the ec2.internal search domain.

   docker compose exec alpaca bash -lc "curl -X GET 'http://houdini:8000/convert?connectionClose=true&disableStreamCache=true'"

   The DNS resolution should now work correctly, resolving domain names without appending unwanted search domains.

   Note: The ping command might resolve when curl does not. Using curl instead of ping is more suitable for testing how PHP applications interact with URLs because curl:

   • Provides a realistic simulation of HTTP/HTTPS requests.
   • Offers detailed diagnostics and supports multiple protocols.
   • Helps identify application-specific issues that ping cannot reveal.

   This makes curl an essential tool for debugging and ensuring proper DNS resolution and connectivity for web applications.

Conclusion

By creating and mounting a custom resolv.conf file in your Docker containers, you can effectively manage DNS settings and avoid issues caused by inherited search domains. This approach provides a consistent and reliable DNS resolution environment for your containerized applications.