AWS IoT - how to use default AWS created RSA certs (ESP32, Sonoff Pow Elite)

[YengaJaf-Aizatron]

I am following the instructions here to connect my ESP32 device to my own AWS backend. I am not using the CloudFormation template, I am doing the whole process manually. I am using a ESP32 devboard for testing and will later use this firmware on a Sonoff Pow R3 Elite. I have successfully compiled and flashed the board. I am able to access the webUI. I have created the AWS Thing as well as the policy.

My issue is that I am having trouble uploading the certificates to the device. The link above uses ECC certificates. I would like to use the default AWS-issued RSA certificates. I am able to upload the root CA1 cert (TLSKey2), but not the private key (TLSKey1). The reason being, I think Tasmota is expecting a 32 byte cert, but I am uploading a larger file.

If anyone could instruct me on how to use AWS-issued RSA certs, I would greatly appreciate it.

Some other questions I had:

• Where are the tlskey files stored? In SPIFFS or in flash? I tried uploading a fake key. I used the one in the link above. After uploading a fake tlskey1, I see there is a tlskey file in the file manager. After uploading tlskey2, I don't see another file in the file manager. Do both keys get stored in the same file, and the keys are then extracted from the single file?
• The default AWS keys seem to have multiple newline characters. Do I need to remove these when uploading them?

[s-hadinger]

Please read carefully the documentation. TLSKey1 is for the device's EC private key (32 bytes, 256 bits). TLSKey2 is for the device certificate signed by AWS IoT.

The root CA is already included in the binary.

[YengaJaf-Aizatron]

My mistake. I was supposed to upload the device certificate to TLSKey2, not the root CA cert. But the point I was trying to make there was that I was able to upload a large certificate file to TLSKey2, and was not able to upload a large certificate file to TLSKey1, and would like some confirmation if it is because the memory space allocated for TLSKey1 is limited to 32 bytes and will not accept anything larger than that. This would seem to be the case.

The main question remains. Is there a way to use the AWS-created device private key (RSA) instead of the smaller EC private key?

[s-hadinger]

No, this was due so storage size constraint on ESP8266. EC private keys are much more compact than RSA keys. And btw they are more secure as well. It is not possible to use RSA keys and devices to authenticate.

Why would you want a RSA cert instead of the smaller and more secure EC cert?

[YengaJaf-Aizatron]

It's just the way we've been doing things in our organisation. It's also easier to just get the certs and keys from AWS as opposed to creating your own. I have never used the AWS CLI, but I guess it's time to learn. I was also not aware that EC keys are more secure than RSA keys. I just assumed since the file size is larger, it would take more work to spoof.

[s-hadinger]

See here: https://www.globalsign.com/en/blog/elliptic-curve-cryptography

256 bits EC key is roughly the strength of 3072 bits RSA.

[s-hadinger]

[YengaJaf-Aizatron]

An extra question, if I want to add the device private key on compilation instead of using the Web UI as instructed in the previous link, where do I put the private key data? The code has this comment on line 449:
// Note: you need to generate a private key + certificate per device and update 'tasmota/tasmota_aws_iot.cpp'
but there is no such file to be found. Since the TLSKey file is stored in SPIFFS, can I create a data folder in the main directory and put the tlskey file which I will programmatically create into that folder?

I do see another comment below that which says:
// for USE_4K_RSA (support for 4096 bits certificates, instead of 2048), you need to uncommend '-DUSE_4K_RSA' in 'build_flags' from 'platform.ini' or 'platform_override.ini'
I have found this line in the platformio.ini file. Is it correct for me to uncomment this so that I can use the AWS-issued RSA device private key? It's a bit misleading because on the Implementation details for AWS-IOT, it says

The server certificate must have an RSA private key (max 2048 bits) and the certificate must be signed with RSA and SHA256 hash. This is the case with default LetsEncrypt certificates. ESP32 supports by default RSA private keys up to 4096 bits, ESP8266 must be compiled with option -DUSE_4K_RSA to support 4096 private keys

This makes me think that I should be able to use the larger RSA private key by default. Or is this note talking about the device certificate (corresponding to TLSKey2), not the device private key?

[s-hadinger]

There is no feature to include the private key on compilation, because it goes against the principle that each device should have its own private key; so burning the key in the firmware is an anti-pattern. But as you mentioned, you can add the file in the filesystem.

The comment
// Note: you need to generate a private key + certificate per device and update 'tasmota/tasmota_aws_iot.cpp' is clearly outdated and obsolete. I will remove it.

The option -DUSE_4K_RSA is now enabled by default. But this refers only to the server's RSA key, not the device's key (which shall be EC)

[s-hadinger]

Since the TLSKey file is stored in SPIFFS, can I create a data folder in the main directory and put the tlskey file which I will programmatically create into that folder?

Yes, you can create the file, and you can embed this file in a factory file, which would flash the ESP32 device with this file already copied in the FS. I haven't done it, but I know it's possible

[YengaJaf-Aizatron]

I have an update on my attempt to connect Tasmota to AWS MQTT.

I've been struggling for quite some time trying to get a connection to AWS MQTT. I kept getting a rc - 2 error. I have the certs uploaded correctly, I configured the AWS policy and things correctly, and used the override settings that I got from AWS-IoT. However, it was not working and I could not figure out why.

I have tried using the GUI WebUI. On the Configure MQTT page, I checked and unchecked this option in the image below, but that did not solve the issue. It actually changed the error from rc -2 to rc -4.

I stumbled across this discussion. One of the comments here say to use SetOption103, which corresponds to the macro definition MQTT_TLS_ENABLED. This macro or SetOptionxxx is not mentioned in the docs. Nevertheless, I tried this and suddenly it worked. I don't understand why using the macro worked, because in my user_config_override.h file, I already had the following:

#ifndef USE_MQTT_TLS
#define USE_MQTT_TLS
#define USE_MQTT_TLS_CA_CERT // Optional but highly recommended
#endif
#ifndef USE_MQTT_AWS_IOT
#define USE_MQTT_AWS_IOT
#endif
#ifdef USE_DISCOVERY
#undef USE_DISCOVERY
#endif

But only when I added:

// corresponds to setoption103
#ifdef MQTT_TLS_ENABLED
#undef MQTT_TLS_ENABLED
#endif
#define MQTT_TLS_ENABLED true

was it able to connect to aws.

Note that my Serial output is not exactly the same as the instructions here.
This is the expected output:

00:00:04 HTP: Web server active on sonoff-4585 with IP address 192.168.1.59
00:00:04 UPP: Multicast (re)joined
21:28:25 MQT: Attempting connection...
21:28:25 MQT: AWS IoT endpoint: xxxxxxxxxxxxx-ats.iot.eu-central-1.amazonaws.com
21:28:26 MQT: AWS IoT connected in 1279 ms
21:28:26 MQT: Connected
21:28:26 MQT: tele/tasmota/LWT = Online
21:28:26 MQT: cmnd/tasmota/POWER =
21:28:26 MQT: tele/tasmota/INFO1 = {"Module":"Sonoff Basic","Version":"6.5.0.14(sonoff)","FallbackTopic":
"cmnd/DVES_67B1E9_fb/","GroupTopic":"sonoffs"}

This is my output (with SerialLog 4):

00:00:15.001 WIF: Connecting to AP1 20BridleCl in mode HT20 as tasmota-07ABF4-3060...
00:00:18.366 WIF: Connected
00:00:18.624 HTP: Web server active on tasmota-07ABF4-3060 with IP address 192.168.1.156
08:50:48.654 MQT: Attempting connection...
08:50:50.490 MQT: TLS connected in 1800 ms, stack low mark 1200
08:50:50.491 MQT: MFLN not supported by TLS server
08:50:50.491 MQT: Connected
08:50:50.506 MQT: tele/tasmota_07ABF4/LWT = Online
08:50:50.509 MQT: cmnd/tasmota_07ABF4/POWER = 
08:50:50.519 MQT: tele/tasmota_07ABF4/INFO1 = {"Info1":{"Module":"ESP32-DevKit","Version":"14.1.0(tasmota32)","FallbackTopic":"cmnd/DVES_07ABF4_fb/","GroupTopic":"cmnd/tasmotas/"}}
08:50:50.535 MQT: tele/tasmota_07ABF4/INFO2 = {"Info2":{"WebServerMode":"Admin","Hostname":"tasmota-07ABF4-3060",...etc}}
08:50:50.550 MQT: tele/tasmota_07ABF4/INFO3 = {"Info3":{"RestartReason":"Vbat power on reset","BootCount":5}}
08:50:53.983 MQT: tele/tasmota_07ABF4/STATE = {"Time":"2024-06-10T08:50:53","Uptime":"0T00:00:24"...etc}

I'm mentioning this on this discussion because I'm not sure if I did something wrong and I just managed to force it to work, or if the docs are needing an update. I am using a MH-ET-LIVE ESP32 minikit devobard. In platformio_override.ini file, under defualt_envs, I am using tasmota32.

[s-hadinger]

Oh you are right. SetOption103 is missing in the doc. It was originally a compile-time option, then moved to have firmwares support both password and certificate support without recompiling.

Your log output is totally fine.