diff --git a/website/content/docs/services/discovery/dns-forwarding/enable.mdx b/website/content/docs/services/discovery/dns-forwarding/enable.mdx new file mode 100644 index 000000000000..b165c2c8fa37 --- /dev/null +++ b/website/content/docs/services/discovery/dns-forwarding/enable.mdx @@ -0,0 +1,420 @@ +--- +layout: docs +page_title: Enable DNS forwarding +description: -> + Learn how to configure different DNS servers to perform DNS forwarding to Consul servers. +--- + +# Enable DNS forwarding + +This page describes the process to enable DNS forwarding to Consul servers. + +You can apply these operations on every node where a Consul agent is running. + +## Requirements + +To enable DNS forwarding, your deployment must have the following: + +- A running Consul server instance +- One or more Consul client nodes with registered services in the Consul catalog +- The `iptables` command available, or one of the following local DNS servers: + - [systemd-resolved](#systemd-resolved) + - [BIND](#bind) + - [Dnsmasq](#dnsmasq) + - [Unbound](#unbound) + - [macOS system resolver](#macos) + +### Network address configuration + +The example configurations on this page assumes Consul's DNS server is listening on the loopback interface on the same node of the local DNS server. + +If Consul is not listening on the loopback IP, replace the references to `localhost` and `120.0.0.1` in the configuration and commands with the appropriate IP address for your environment. + +## systemd-resolved + +[`systemd-resolved`](https://www.freedesktop.org/software/systemd/man/latest/systemd-resolved.service.html) is a system service that provides network name resolution to local applications. It is the default local DNS server for many Linux distributions. + +To configure the `systemd-resolved` service so that it sends `.consul` domain queries to Consul, create a `consul.conf` file located in the `/etc/systemd/resolved.conf.d/` directory. + + + + +Add a `[Resolve]` section to your resolved configuration. + + + +```ini +[Resolve] +DNS=127.0.0.1 +DNSSEC=false +Domains=~consul +``` + + + +### Define port for Consul DNS server + +When using systemd 245 and older, you cannot specify port numbers in the `DNS` configuration field. systemd-resolved only uses port `53`, which is a privileged port. + +When you cannot specify ports for the system's configuration, there are two workarounds: + - [Configure Consul DNS service to listen on port `53`](/consul/docs/agent/config/config-files#dns_port) instead of `8600`. + - Map port `53` to `8600` using `iptables`. + +Binding to port `53` usually requires running Consul as a privileged user or running Linux with the `CAP_NET_BIND_SERVICE` capability. +When using the Consul Docker image, add the following to the environment to allow Consul to use the port: `CONSUL_ALLOW_PRIVILEGED_PORTS=yes`. + +To avoid running Consul as a privileged user, the following `iptables` commands are sufficient to map port `53` to `8600` and redirect DNS queries to Consul. + +```shell-session +# iptables --table nat --append OUTPUT --destination localhost --protocol udp --match udp --dport 53 --jump REDIRECT --to-ports 8600 \ + iptables --table nat --append OUTPUT --destination localhost --protocol tcp --match tcp --dport 53 --jump REDIRECT --to-ports 8600 +``` + + + + + +Systemd 246 and newer allow you to specify the DNS port directly in the `systemd-resolved` configuration file. +Previous versions of systemd required iptables rules to direct DNS traffic to Consul. + +Add a `[Resolve]` section to your resolved configuration. + + + +```ini +[Resolve] +DNS=127.0.0.1:8600 +DNSSEC=false +Domains=~consul +``` + + + + + + +PTR record queries are still sent to the other configured resolvers, in addition to Consul. + +After creating the resolved configuration, restart `systemd-resolved`. + +```shell-session +# systemctl restart systemd-resolved +``` + +The command produces no output. + +### Validate the systemd-resolved configuration + +Validate that `systemd-resolved` is active. + +```shell-session +# systemctl is-active systemd-resolved +active +``` + +Verify that `systemd-resolved` is configured to forward queries for the `consul` domain to Consul. + +```shell-session +# resolvectl domain +Global: ~consul +Link 2 (eth0): +``` + +Verify that `systemd-resolved` is able to resolve the Consul server address. + +```shell-session +# resolvectl query consul.service.consul +consul.service.consul: 127.0.0.1 + +-- Information acquired via protocol DNS in 6.6ms. +-- Data is authenticated: no +``` + +Confirm that `/etc/resolv.conf` points to the `stub-resolv.conf` file managed by `systemd-resolved`. + +```shell-session +$ ls -l /etc/resolv.conf +lrwxrwxrwx 1 root root 37 Jul 14 10:10 /etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf +``` + +Confirm that the IP address for `systemd-resolved`'s stub resolver is the configured `nameserver`. + + + +```shell-session +$ cat /etc/resolv.conf +## This file is managed by man:systemd-resolved(8). Do not edit. +## +## This is a dynamic resolv.conf file for connecting local clients to the +## internal DNS stub resolver of systemd-resolved. This file lists all +## configured search domains. +## +## Run "resolvectl status" to see details about the uplink DNS servers +## currently in use. +## +## Third party programs must not access this file directly, but only through the +## symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way, +## replace this symlink by a static file or a different symlink. +## +## See man:systemd-resolved.service(8) for details about the supported modes of +## operation for /etc/resolv.conf. + +nameserver 127.0.0.53 +options edns0 +``` + + + +Ensure that the operating system can resolve DNS queries to the `.consul` domain. + +```shell-session +$ host consul.service.consul +consul.service.consul has address 127.0.0.1 +``` + +### Using any local resolver with systemd + +By default, the local resolver stub in the `resolved.conf` file is configured to listen for UDP and TCP requests at `127.0.0.53:53`. However, you can set the `DNSStubListener` option to `false` so that your system can use any DNS configuration, as long as it loads earlier than `resolved`. + + + +```plaintext +DNSStubListener=false +``` + + + +## Dnsmasq + +Use [dnsmasq](http://www.thekelleys.org.uk/dnsmasq/doc.html) if you have a small network and need a lightweight DNS solution. + + + +If your distribution uses systemd, disable `systemd-resolved` before you follow these steps. + + + +Configure the `dnsmasq.conf` file or a series of files in the `/etc/dnsmasq.d` directory. Add server settings to your configuration file so that requests for the `consul` domain are forwarded to Consul DNS. + + + +```plaintext +# Enable forward lookup of the 'consul' domain: +server=/consul/127.0.0.1#8600 + +# Uncomment and modify as appropriate to enable reverse DNS lookups for +# common netblocks found in RFC 1918, 5735, and 6598: +#rev-server=0.0.0.0/8,127.0.0.1#8600 +#rev-server=10.0.0.0/8,127.0.0.1#8600 +#rev-server=100.64.0.0/10,127.0.0.1#8600 +#rev-server=127.0.0.1/8,127.0.0.1#8600 +#rev-server=169.254.0.0/16,127.0.0.1#8600 +#rev-server=172.16.0.0/12,127.0.0.1#8600 +#rev-server=192.168.0.0/16,127.0.0.1#8600 +#rev-server=224.0.0.0/4,127.0.0.1#8600 +#rev-server=240.0.0.0/4,127.0.0.1#8600 +# Accept DNS queries only from hosts whose address is on a local subnet. +#local-service +# Don't poll /etc/resolv.conf for changes. +#no-poll +# Don't read /etc/resolv.conf. Get upstream servers only from the command +# line or the dnsmasq configuration file (see the "server" directive below). +#no-resolv +# Specify IP address(es) of other DNS servers for queries not handled +# directly by consul. There is normally one 'server' entry set for every +# 'nameserver' parameter found in '/etc/resolv.conf'. See dnsmasq(8)'s +# 'server' configuration option for details. +#server=1.2.3.4 +#server=208.67.222.222 +#server=8.8.8.8 +# Set the size of dnsmasq's cache. The default is 150 names. Setting the +# cache size to zero disables caching. +#cache-size=65536 +``` + + + +Restart the `dnsmasq` service after creating the configuration. + +Refer to [`dnsmasq(8)`](http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html) for additional configuration settings such as specifying IP addresses for queries not handled directly by Consul. + + +## BIND + +[BIND](https://www.isc.org/bind/) is a robust DNS system. Its most prominent component, `named`, performs both of the main DNS server roles, acts as an authoritative name server for DNS zones, and is a recursive resolver in the network. + + + +If your distribution uses systemd, disable `systemd-resolved` before you follow these steps. + + + +To configure the BIND service to send `.consul` domain queries to Consul: + +1. Create a `named` configuration file with `DNSSEC` disabled. +1. Create a zone configuration file to manage the `.consul` domain. + +### Named configuration file + +Edit the `/etc/named.conf` to configure your BIND instance. Remember to disable `DNSSEC` so that Consul and BIND can communicate. Add an `include` section to include the zone file that you create in the next step. + +The following example shows a BIND configuration with `DNSSEC` disabled. + + + +```plaintext +options { + listen-on port 53 { 127.0.0.1; }; + listen-on-v6 port 53 { ::1; }; + directory "/var/named"; + dump-file "/var/named/data/cache_dump.db"; + statistics-file "/var/named/data/named_stats.txt"; + memstatistics-file "/var/named/data/named_mem_stats.txt"; + allow-query { localhost; }; + recursion yes; + + dnssec-enable no; + dnssec-validation no; + + /* Path to ISC DLV key */ + bindkeys-file "/etc/named.iscdlv.key"; + + managed-keys-directory "/var/named/dynamic"; +}; + +include "/etc/named/consul.conf"; +``` + + + +### Zone configuration file + +Set up a zone for your Consul-managed records in `consul.conf`. + + + +```dns-zone-file +zone "consul" IN { + type forward; + forward only; + forwarders { 127.0.0.1 port 8600; }; +}; +``` + + + +## Unbound + +Use [Unbound](https://www.unbound.net/) when you need a fast and lean DNS resolver for Linux and macOS. + + + +If your distribution uses systemd, disable `systemd-resolved` before you follow these steps. + + + + +The following example demonstrates a configuration for the `consul.conf` file in the `/etc/unbound/unbound.conf.d` directory. + +Add `server` and `stub-zone` settings to your Unbound configuration file. + + + +```plaintext +#Allow insecure queries to local resolvers +server: + do-not-query-localhost: no + domain-insecure: "consul" + +#Add consul as a stub-zone +stub-zone: + name: "consul" + stub-addr: 127.0.0.1@8600 +``` + + + +You may have to add the following line to the bottom of your `/etc/unbound/unbound.conf` file for the new configuration to be included. + + + +```plaintext +include: "/etc/unbound/unbound.conf.d/*.conf" +``` + + + +## iptables + +[iptables](https://www.netfilter.org/projects/iptables/index.html) is a generic firewalling software that allows you to define traffic rules for your system. + +If you do not have a local DNS server on the Consul agent node, use `iptables` to forward DNS requests on port `53` to the Consul agent running on the same machine without using a secondary service. + +This configuration realizes full DNS forwarding, which means that all DNS queries for the host are forwarded to Consul, not just the ones for the `.consul` top level domain. Consul's default configuration resolves only the `.consul` top level domain, so you must set the [`recursors`](/consul/docs/agent/config/config-files#recursors) flag if you want your node to be able to resolve other domains when using `iptables` configuration. + +If you use DNS relay hosts in your network, do not place them on the same host as Consul. The redirects may intercept the traffic. + +### Configure Consul recursors + +Add recursors to your Consul configuration. + + + +```hcl +# DNS recursors +recursors = [ "1.1.1.1" ] +``` + + + +Recursors should not include the `localhost` address because the `iptables` redirects would intercept the requests. + +You can replace the `1.1.1.1` address in the example with another DNS server address. This is suitable for situations where an external DNS +service is already running in your infrastructure and is used as the recursor. + +### Create iptables rules + +After you configure Consul to use a valid recursor, add rules to `iptables` to redirect traffic from port `53` to port `8600`. + +```shell-session +# iptables -t nat -A PREROUTING -p udp -m udp --dport 53 -j REDIRECT --to-ports 8600 \ + iptables -t nat -A PREROUTING -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 8600 \ + iptables -t nat -A OUTPUT -d localhost -p udp -m udp --dport 53 -j REDIRECT --to-ports 8600 \ + iptables -t nat -A OUTPUT -d localhost -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 8600 +``` + +## macOS + +On macOS systems, use the macOS system resolver to point all `.consul` requests to Consul. +The `man 5 resolver` command describes this feature in more detail. + +The following instructions require `sudo` or root access. + +To configure the macOS system resolver to forward DNS queries to Consul, add a resolver entry in the `/etc/resolver/` directory that points at the Consul agent. + +If you do not have this folder, create it. + +```shell-session +# mkdir -p /etc/resolver +``` + +Create a new file `/etc/resolver/consul` with `nameserver` and `port` entries. + + + +```plaintext +nameserver 127.0.0.1 +port 8600 +``` + + + +The configuration informs the macOS resolver daemon to forward all `.consul` TLD requests to `127.0.0.1` on port `8600`. + +## Next steps + +This instructions on this page helped you configure your node to forward DNS requests to Consul. + +To learn more on how to query Consul DNS once forwarding is enabled, refer to [DNS forwarding workflow](/consul/docs/services/discovery/dns-forwarding#workflow). + +For more information on other DNS features and configurations available in Consul, refer to [DNS usage overview](/consul/docs/services/discovery/dns-overview). \ No newline at end of file diff --git a/website/content/docs/services/discovery/dns-forwarding/index.mdx b/website/content/docs/services/discovery/dns-forwarding/index.mdx new file mode 100644 index 000000000000..1ee28417397f --- /dev/null +++ b/website/content/docs/services/discovery/dns-forwarding/index.mdx @@ -0,0 +1,186 @@ +--- +layout: docs +page_title: DNS forwarding +description: -> + Learn how to configure your local DNS servers to perform DNS forwarding to Consul servers. +--- + +# DNS forwarding + +This topic describes the process to configure different DNS servers to enable DNS forwarding to Consul servers. + +You can apply these operations on every node where a Consul agent is running. + +## Introduction + +You deployed a Consul datacenter and want to use Consul DNS interface for name resolution. + +When configured with default values, Consul exposes the DNS interface on port `8600`. By default, DNS is served from port `53`. On most operating systems, this requires elevated privileges. It is also common, for most operating systems, to have a local DNS server already running on port `53`. + +Instead of running Consul with an administrative or root account, you can forward appropriate queries to Consul, running on an unprivileged port, from another DNS server or using port redirect. + +There are two configurations for a node's DNS forwarding behavior: + - **Conditional DNS forwarding**: the local DNS servers are configured to forward to Consul only queries relative to the `.consul` zone. All other queries are still served via the default DNS server in the node. + - **Full DNS forwarding**: Consul serves all DNS queries and forwards to a remote DNS server the ones outside `.consul` domain. + +### Conditional DNS forwarding + +We recommend the conditional DNS forwarding approach. This configuration lowers the Consul agent's resource consumption by limiting the number of DNS requests it handles. + +![Consul DNS conditional forwarding - Only .consul requests are routed to Consul](/img/consul-dns-conditional-forwarding.png#light-theme-only) +![Consul DNS conditional forwarding - Only .consul requests are routed to Consul](/img/consul-dns-conditional-forwarding-dark.png#dark-theme-only) + +In this configuration, Consul only serves queries relative to the `.consul` domain. There is no unnecessary load on Consul servers to serve queries from different domains. + +This behavior is not enabled by default. + +### Full DNS forwarding + +This approach can be useful in scenarios where the Consul agent's node is allocated limited resources and you want to avoid the overhead of running a local DNS server. In this configuration, Consul serves all DNS queries for all domains and forwards the ones outside the `.consul` domain to one or more configured forwarder servers. + +![Consul DNS forwarding - All requests are routed to Consul](/img/consul-dns-forwarding.png#light-theme-only) +![Consul DNS forwarding - All requests are routed to Consul](/img/consul-dns-forwarding-dark.png#dark-theme-only) + +This behavior is not enabled by default. Consul standard configuration only resolves DNS records inside the `.consul` zone. To enable DNS forwarding, you need to set the [recursors](/consul/docs/agent/config/config-files#recursors) option in your Consul configuration. + +In this scenario, if a Consul DNS reply includes a `CNAME` record pointing outside the `.consul` top level domain, then the DNS reply only includes `CNAME` records by default. + +When `recursors` is set and the upstream resolver is functioning correctly, Consul tries to resolve CNAMEs and include any records (for example, A, AAAA, PTR) for them in its DNS reply. In these scenarios, Consul is used for full DNS forwarding and is able to serve queries for all domains. + +## Workflow + +To use DNS forwarding in Consul deployments, complete the following steps: + +1. Configure the local DNS service to enable DNS forwarding to Consul. Follow the instructions for one of the following services: + + - [systemd-resolved](/consul/docs/services/discovery/dns-forwarding/enable#systemd-resolved) + - [BIND](/consul/docs/services/discovery/dns-forwarding/enable#bind) + - [Dnsmasq](/consul/docs/services/discovery/dns-forwarding/enable#dnsmasq) + - [Unbound](/consul/docs/services/discovery/dns-forwarding/enable#unbound) + - [iptables](/consul/docs/services/discovery/dns-forwarding/enable#iptables) + - [macOS system resolver](/consul/docs/services/discovery/dns-forwarding/enable#macOS) + +1. Query the Consul DNS to confirm that DNS forwarding functions correctly. + + ```shell-session + $ dig consul.service.consul A + + ; <<>> DiG 9.16.48-Debian <<>> consul.service.consul A + ;; global options: +cmd + ;; Got answer: + ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51736 + ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1 + + ;; OPT PSEUDOSECTION: + ; EDNS: version: 0, flags:; udp: 65494 + ;; QUESTION SECTION: + ;consul.service.consul. IN A + + ;; ANSWER SECTION: + consul.service.consul. 0 IN A 10.0.4.140 + consul.service.consul. 0 IN A 10.0.4.121 + consul.service.consul. 0 IN A 10.0.4.9 + + ;; Query time: 4 msec + ;; SERVER: 127.0.0.53#53(127.0.0.53) + ;; WHEN: Wed Jun 26 20:47:05 UTC 2024 + ;; MSG SIZE rcvd: 98 + + ``` + +1. Optionally, verify reverse DNS. + + ```shell-session + $ dig 140.4.0.10.in-addr.arpa. PTR + + ; <<>> DiG 9.16.48-Debian <<>> 140.4.0.10.in-addr.arpa. PTR + ;; global options: +cmd + ;; Got answer: + ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35085 + ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 + + ;; OPT PSEUDOSECTION: + ; EDNS: version: 0, flags:; udp: 65494 + ;; QUESTION SECTION: + ;140.4.0.10.in-addr.arpa. IN PTR + + ;; ANSWER SECTION: + 140.4.0.10.in-addr.arpa. 0 IN PTR consul-server-0.node.dc1.consul. + + ;; Query time: 0 msec + ;; SERVER: 127.0.0.53#53(127.0.0.53) + ;; WHEN: Wed Jun 26 20:47:57 UTC 2024 + ;; MSG SIZE rcvd: 97 + + ``` + + You can use the `short` option for `dig` to only get the node name instead of the full output. + + ```shell-session + $ dig +short -x 10.0.4.140 + consul-server-0.node.dc1.consul. + ``` + +## Troubleshooting + +If your DNS server does not respond but you do get an answer from Consul, turn on your DNS server's query log to check for errors. + +### systemd-resolved + +Enable query logging for `systemd-resolved`: + +```shell-session +# resolvectl log-level debug +``` + +Check query log: + +```shell-session +# journalctl -r -u systemd-resolved +``` + +Disable query logging: + +```shell-session +# resolvectl log-level info +``` + +DNS forwarding may fail if you use the default `systemd-resolved` configuration and attempt to bind to `0.0.0.0`. The default configuration uses a DNS stub that listens for UDP and TCP requests at `127.0.0.53`. As a result, attempting to bind to `127.0.0.53` conflicts with the running stub. You can disable the stub as described in the [Using any local resolver with systemd](/consul/docs/services/discovery/dns-forwarding/enable#using-any-local-resolver-with-systemd) section to troubleshoot this problem. + +### Dnsmasq + +To enable query log refer to [Dnsmasq documentation](https://thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html). + +In particular, look for the `log-queries` and `log-facility` configuration option. + +When query log is enabled, it is possible to force Dnsmasq to emit a full cache dump using the `SIGUSR1` signal. + +### BIND + +Enable query log: + +```shell-session +$ rndc querylog +``` + +Check logs: + +```shell-session +$ tail -f /var/log/messages +``` + +The log may show errors like this: + + + +```plaintext +error (no valid RRSIG) resolving +error (no valid DS) resolving +``` + + + +This error indicates that `DNSSEC` is not disabled properly. + +If you receive errors about network connections, verify that there are no firewall +or routing problems between the servers running BIND and Consul. diff --git a/website/content/docs/services/discovery/dns-overview.mdx b/website/content/docs/services/discovery/dns-overview.mdx index 34e92e5fe8a2..dd2b22bcf113 100644 --- a/website/content/docs/services/discovery/dns-overview.mdx +++ b/website/content/docs/services/discovery/dns-overview.mdx @@ -30,6 +30,12 @@ If you are using Consul for service mesh on VMs, you can use upstreams or DNS. W If you are using Consul on Kubernetes, refer to [the upstreams annotation documentation](/consul/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) for additional information. +## DNS forwarding + +You can configure your local DNS servers to use Consul. + +Refer to [DNS Forwarding](/consul/docs/services/discovery/dns-forwarding) for additional information. + ## Static queries Node lookups and service lookups are the fundamental types of static queries. Depending on your use case, you may need to use different query methods and syntaxes to query the DNS for services and nodes. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 38126b547e4c..c883f29a5bec 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -389,6 +389,19 @@ "title": "Configure DNS behavior", "path": "services/discovery/dns-configuration" }, + { + "title": "DNS forwarding", + "routes": [ + { + "title": "Overview", + "path": "services/discovery/dns-forwarding" + }, + { + "title": "Enable DNS forwarding", + "path": "services/discovery/dns-forwarding/enable" + } + ] + }, { "title": "Perform static DNS lookups", "path": "services/discovery/dns-static-lookups" diff --git a/website/public/img/consul-dns-conditional-forwarding-dark.png b/website/public/img/consul-dns-conditional-forwarding-dark.png new file mode 100644 index 000000000000..b54ed817eb6c Binary files /dev/null and b/website/public/img/consul-dns-conditional-forwarding-dark.png differ diff --git a/website/public/img/consul-dns-conditional-forwarding.png b/website/public/img/consul-dns-conditional-forwarding.png new file mode 100644 index 000000000000..82ea08a11b2a Binary files /dev/null and b/website/public/img/consul-dns-conditional-forwarding.png differ diff --git a/website/public/img/consul-dns-forwarding-dark.png b/website/public/img/consul-dns-forwarding-dark.png new file mode 100644 index 000000000000..9ff518397b81 Binary files /dev/null and b/website/public/img/consul-dns-forwarding-dark.png differ diff --git a/website/public/img/consul-dns-forwarding.png b/website/public/img/consul-dns-forwarding.png new file mode 100644 index 000000000000..bffce6e333a9 Binary files /dev/null and b/website/public/img/consul-dns-forwarding.png differ