Strip trailing dot from FQDNs in Host and TLS

The TLS verification fails with an exception if the client uses
a fully-qualified domain name with a trailing dot,
like https://github.com./ :

aiohttp.client_exceptions.ClientConnectorCertificateError:
Cannot connect to host github.com.:443 ssl:True
[SSLCertVerificationError: (1, "[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: Hostname mismatch,
certificate is not valid for 'github.com.'. (_ssl.c:1051)")]

The reason is that TLS certificates do not contain the trailing dot,
as per RFC 6066:

"HostName" contains the fully qualified DNS hostname of the server,
   as understood by the client.  The hostname is represented as a byte
   string using ASCII encoding without a trailing dot.

We need to strip the trailing dot for TLS context and Host header,
where trailing dots are not present.
For DNS resolution, we need to include the trailing dot as it signifies
a fully-qualified domain name (FQDN).
DNS lookups of FQDNs are faster as the resolver does not need to check
DNS search path, like for relative DNS names.

Closes #3636

CHANGES/3636.bugfix

@@ -0,0 +1,2 @@
+Strip trailing dot in Host header and TLS context from fully-qualified domain names.
+This allows the client to connect to URLs with FQDN hostname like `https://example.com./`

CONTRIBUTORS.txt

@@ -221,6 +221,7 @@ Marko Kohtala
 Martijn Pieters
 Martin Melka
 Martin Richard
+Martin Sucha
 Mathias Fröjdman
 Mathieu Dugré
 Matt VanEseltine

aiohttp/client_reqrep.py

@@ -340,6 +340,9 @@ def update_headers(self, headers: Optional[LooseHeaders]) -> None:
         netloc = cast(str, self.url.raw_host)
         if helpers.is_ipv6_address(netloc):
             netloc = f"[{netloc}]"
+        elif netloc.endswith(":"):
+            # Strip trailing dot. See https://github.com/aio-libs/aiohttp/issues/3636.
+            netloc = netloc[:-1]
         if self.url.port is not None and not self.url.is_default_port():
             netloc += ":" + str(self.url.port)
         self.headers[hdrs.HOST] = netloc

aiohttp/connector.py

@@ -1117,6 +1117,13 @@ def drop_exception(fut: "asyncio.Future[List[Dict[str, Any]]]") -> None:
             port = hinfo["port"]
 
             try:
+                server_hostname = None
+                if sslcontext:
+                    server_hostname = hinfo["hostname"]
+                    if server_hostname.endswith("."):
+                        # Strip trailing dot, certificates contain FQDN without dot.
+                        # See https://github.com/aio-libs/aiohttp/issues/3636
+                        server_hostname = server_hostname[:-1]
                 transp, proto = await self._wrap_create_connection(
                     self._factory,
                     host,
@@ -1126,7 +1133,7 @@ def drop_exception(fut: "asyncio.Future[List[Dict[str, Any]]]") -> None:
                     family=hinfo["family"],
                     proto=hinfo["proto"],
                     flags=hinfo["flags"],
-                    server_hostname=hinfo["hostname"] if sslcontext else None,
+                    server_hostname=server_hostname,
                     local_addr=self._local_addr,
                     req=req,
                     client_error=client_error,