Specify SNI server_hostname when performing request with asyncio/aiohttp

[AsyncBrain]

I also posted this HERE

The context :

            load-balancer.example.com:443 (TCP passthrough)
                               /\
                              /  \
                             /    \
                            /      \
          s1.example.com:443        s2.example.com:443
              (SSL/SNI)                  (SSL/SNI)              

I am able to launch a huge batch of async ClientSession.get requests on both upstreams in parallel but for each request I need to somehow tell asyncio or aiohttp to use load-balancer.example.com as server_hostname, otherwise the SSL handshake fails.

Is there an easy way to setup the ClientSession to use a specific server_hostname when setting up the SSL socket ?
Does someone have already done something like that ?

The goal is to stress-test the upstreams s1 and s2 directly using aiohttp with certificate-validation enable. Since the load-balancer does not belong to me I don't want to do the stress-test over it.

• the code is not supposed to run on other platforms than GNU Linux with at least Python-v3.7 (but I can use any recent version if needed)
• all servers serve a valid certificate for load-balancer.example.com
• openssl validates the certificate from the upstreams when using openssl s_connect s1.example.com:443 -servername load-balancer.example.com
• cURL needs curl 'https://load-balancer.example.com/' --resolve s1.example.com:443:load-balancer.example.com and also validates successfully

here is the most simple snippet with just a single request :

import aiohttp
import asyncio


async def main_async(host, port, uri, params=[], headers={}, sni_hostname=None):
    if sni_hostname is not None:
        print('Setting SNI server_name field ')
        #
        # THIS IS WHERE I DON'T KNOW HOW TO TELL ASYNCIO
        # TO SET THE SERVER_NAME FIELD TO sni_hostname
        # IN THE SSL SOCKET BEFORE PERFORMING THE SSL HANDSHAKE
        #
    try:
        async with aiohttp.ClientSession(raise_for_status=True) as session:
            async with session.get(f'https://{host}:{port}/{uri}', params=params, headers=headers) as r:
                body = await r.read()
                print(body)
    except Exception as e:
        print(f'Exception while requesting ({e}) ')


if __name__ == "__main__":
    asyncio.run(main_async(host='s1.example.com', port=443,
                           uri='/api/some/endpoint',
                           params={'apikey': '0123456789'},
                           headers={'Host': 'load-balancer.example.com'},
                           sni_hostname='load-balancer.example.com'))

When running it with real hosts, it throws

Cannot connect to host s1.example.com:443 ssl:True 
[SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] '
certificate verify failed: certificate has expired (_ssl.c:1131)')]) 

Note that the error certificate has expired indicates that the certificate proposed to the client is the default certificate since the SNI hostname is s1.example.com which is unknow by the webserver running there.

When running it against the load-balancer it works just fine, the SSL handshake happens with the upstreams which serve the certificate and everything is valid.

Also note that

• sni_callback does not help since it is called after the handshake has started and the certificate was received (and at this point server_hostname is a read-only property anyway)
• it does not seem to be possible to set server_hostname when creating an SSLContext allthough SSLContext.wrap_socket does support server_hostname but I was not able to make that work

I hope someone knows how to fill the comment block in that snippet ;-]

Related component

Client