Test failure: System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors(url: \"https://self-signed.badssl.com/\", expectedErrors: RemoteCertificateChainErrors)

[v-haren]

failed in job: runtime-libraries outerloop 20200825.2

net5.0-OSX-Release-x64-CoreCLR_release-OSX.1015.Amd64.Open

Error message

System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception.
---- Assert.Equal() Failure
Expected: RemoteCertificateChainErrors
Actual:   RemoteCertificateNameMismatch | RemoteCertificateChainErrors


Stack trace
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 140
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1293
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1327
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 536
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 869
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
   at System.Net.Http.HttpClient.FinishSendAsync(ValueTask`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts, Boolean buffered, Boolean async, CancellationToken callerToken, Int64 timeoutTime) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 605
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors_Helper(String url, String useHttp2String, SslPolicyErrors expectedErrors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 334
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors(String url, SslPolicyErrors expectedErrors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 357
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.<>c__DisplayClass18_0.<UseCallback_BadCertificate_ExpectedPolicyErrors_Helper>b__0(HttpRequestMessage request, X509Certificate2 cert, X509Chain chain, SslPolicyErrors errors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 330
   at System.Net.Http.ConnectHelper.<>c__DisplayClass3_0.<EstablishSslConnectionAsync>b__0(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 98
   at System.Net.Security.SecureChannel.VerifyRemoteCertificate(RemoteCertificateValidationCallback remoteCertValidationCallback, ProtocolToken& alertToken, SslPolicyErrors& sslPolicyErrors, X509ChainStatusFlags& chainStatus) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs:line 989
   at System.Net.Security.SslStream.CompleteHandshake(ProtocolToken& alertToken, SslPolicyErrors& sslPolicyErrors, X509ChainStatusFlags& chainStatus) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 612
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 410
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 116

[ghost]

Tagging subscribers to this area: @safern, @ViktorHofer
See info in area-owners.md if you want to be subscribed.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[safern]

cc: @dotnet/ncl

[karelz]

@wfurt any thoughts?

[wfurt]

I have local repro. I will take a look. maybe something changed at self-signed.badssl.com.

[wfurt]

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 14824823351240255409 (0xcdbc5a4aec9767b1)
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=California, L=San Francisco, O=BadSSL, CN=BadSSL Intermediate Certificate Authority
        Validity
            Not Before: Aug  8 21:17:05 2016 GMT
            Not After : Aug  8 21:17:05 2018 GMT
        Subject: C=US, ST=California, L=San Francisco, O=BadSSL Fallback. Unknown subdomain or no SNI., CN=badssl-fallback-unknown-subdomain-or-no-sni
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:c2:04:ec:f8:8c:ee:04:c2:b3:d8:50:d5:70:58:
                    cc:93:18:eb:5c:a8:68:49:b0:22:b5:f9:95:9e:b1:
                    2b:2c:76:3e:6c:c0:4b:60:4c:4c:ea:b2:b4:c0:0f:
                    80:b6:b0:f9:72:c9:86:02:f9:5c:41:5d:13:2b:7f:
                    71:c4:4b:bc:e9:94:2e:50:37:a6:67:1c:61:8c:f6:
                    41:42:c5:46:d3:16:87:27:9f:74:eb:0a:9d:11:52:
                    26:21:73:6c:84:4c:79:55:e4:d1:6b:e8:06:3d:48:
                    15:52:ad:b3:28:db:aa:ff:6e:ff:60:95:4a:77:6b:
                    39:f1:24:d1:31:b6:dd:4d:c0:c4:fc:53:b9:6d:42:
                    ad:b5:7c:fe:ae:f5:15:d2:33:48:e7:22:71:c7:c2:
                    14:7a:6c:28:ea:37:4a:df:ea:6c:b5:72:b4:7e:5a:
                    a2:16:dc:69:b1:57:44:db:0a:12:ab:de:c3:0f:47:
                    74:5c:41:22:e1:9a:f9:1b:93:e6:ad:22:06:29:2e:
                    b1:ba:49:1c:0c:27:9e:a3:fb:8b:f7:40:72:00:ac:
                    92:08:d9:8c:57:84:53:81:05:cb:e6:fe:6b:54:98:
                    40:27:85:c7:10:bb:73:70:ef:69:18:41:07:45:55:
                    7c:f9:64:3f:3d:2c:c3:a9:7c:eb:93:1a:4c:86:d1:
                    ca:85
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Basic Constraints:
                CA:FALSE
            X509v3 Subject Alternative Name:
                DNS:badssl-fallback-unknown-subdomain-or-no-sni

it seems fresh and the name does not match. It is curious that it fails only on OSX.
I did run on Linux where I would expect same failure but it seems ok.

cc: @bartonjs for any additional thoughts.

since this is HTTP Client test, perhaps we can construct this with SslStream without reliance on external server we cannot control.

[bartonjs]

Weird, it looks like macOS just didn't do SNI.

I think this test was written prior to CertificateRequest existing, so creating self signed certs had meant a) check one in and wait for it to expire or b) depend on an external resource. Now that certs can be easily made on the fly, I agree that it can go from an external test to a self-test.

[wfurt]

my bad @bartonjs. I got the posted certificate when running openssl from command line. When I run the test, SNI is set and test gets different certificate. That on is for *.badssl.com so it is curious why OSX would complain about the name.

[wfurt]

It seems like this is essentially dup of #666. The verification fails with CSSMERR_APPLETP_INVALID_EXTENDED_KEY_USAGE because the self-signed certificate does not have EKU set.

The question is if EKU should cause RemoteCertificateNameMismatch...? (I think RemoteCertificateChainErrors would still make sense if OS cares about EKU)
Or should `AppleCryptoNative_SslIsHostnameMatch focus on name?

cc: @vcsjones

[bartonjs]

Ah. That.

The way we currently detect a hostname mismatch on macOS is to build their version of X509Chain but with an "I'm doing TLS" policy applied. The assumption is that if the base policy passes, but the TLS policy fails, that it failed because of the hostname; but apparently there are secondary characteristics that can also cause it to fail.

So there's a tradeoff: a few false negatives, or all of the various bugs/complaints that come from trying to write our own version of the wildcard matching. (It's so much easier to say "it's an OS problem", but it's double-so-much-easier when that's exposed directly.)

Maybe that functionality has become more public/directly-diagnosable since the original implementation.

[wfurt]

I will rewrite the test to use certificate we can control to make it less dependent on external entities.

We already have enough hooks that we could ignore specific error codes during name validation e.g. use what you described with exceptions. I would expect that we validate rest of the certificate outside of _SslIsHostnameMatch ...?
Or are we not?

[bartonjs]

I would expect that we validate rest of the certificate outside of _SslIsHostnameMatch ...?

The X509Chain class is used to determine the RemoteCertificateChainErrors value, and to pass some context into the AppleCryptoNative_SslIsHostnameMatch function in order to build the request (e.g. it always says that the context is trusted so that an untrusted chain doesn't produce hostname mismatch).

Trying to tear apart the false to distinguish "name mismatch" specifically from "generically not valid for TLS" has virtue, but I don't think it'd be approved for 5.0 at this point as it's not a regression.

[v-haren]

failed again in job: runtime-libraries outerloop 20200902.4

failed test: System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors(url: "https://self-signed.badssl.com/\", expectedErrors: RemoteCertificateChainErrors)

net5.0-OSX-Release-x64-CoreCLR_release-OSX.1015.Amd64.Open

Error message

System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception.
---- Assert.Equal() Failure\nExpected: RemoteCertificateChainErrors
Actual:   RemoteCertificateNameMismatch | RemoteCertificateChainErrors


Stack trace
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 138
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1287
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 1324
   at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 548
   at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs:line 863
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/RedirectHandler.cs:line 30
   at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs:line 736
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors_Helper(String url, String useHttp2String, SslPolicyErrors expectedErrors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 334
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.UseCallback_BadCertificate_ExpectedPolicyErrors(String url, SslPolicyErrors expectedErrors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 357
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
   at System.Net.Http.Functional.Tests.HttpClientHandler_ServerCertificates_Test.<>c__DisplayClass18_0.<UseCallback_BadCertificate_ExpectedPolicyErrors_Helper>b__0(HttpRequestMessage request, X509Certificate2 cert, X509Chain chain, SslPolicyErrors errors) in /_/src/libraries/Common/tests/System/Net/Http/HttpClientHandlerTest.ServerCertificates.cs:line 330
   at System.Net.Http.ConnectHelper.<>c__DisplayClass3_0.<EstablishSslConnectionAsync>b__0(Object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 85
   at System.Net.Security.SecureChannel.VerifyRemoteCertificate(RemoteCertificateValidationCallback remoteCertValidationCallback, ProtocolToken& alertToken, SslPolicyErrors& sslPolicyErrors, X509ChainStatusFlags& chainStatus) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs:line 995
   at System.Net.Security.SslStream.CompleteHandshake(ProtocolToken& alertToken, SslPolicyErrors& sslPolicyErrors, X509ChainStatusFlags& chainStatus) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 619
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 410
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Boolean async, Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/ConnectHelper.cs:line 138