HTTPUpdate read Timeout

[hitecSmartHome]

Board

ESP32-Wrover

Device Description

Hardware Configuration

Version

v2.0.17

IDE Name

PlatformIO

Operating System

Windows10

Flash frequency

80

PSRAM enabled

yes

Upload speed

115200

Description

I want to perform a HTTPUpdate which was working fine before but suddenly I got read Timeout.

Sketch

WiFiClientSecure secureClient;
HTTPUpdate Updater;

secureClient.setInsecure();
secureClient.setTimeout(35);

Updater.rebootOnUpdate(false);
Updater.setFollowRedirects(HTTPC_FORCE_FOLLOW_REDIRECTS);

Updater.onStart([this]() { updateStart(); });
Updater.onEnd([this]() { updateSuccess(); });
Updater.onError([this](int err) { updateError(err); });
Updater.onProgress([this](int current, int total) {
    updateProgressReport(100.0 * current / total);
});

void VersionHandler::startFirmware(){
    updateStage = FIRMWARE_STAGE;
    std::string url = std::string(config.serverBaseURL) + "version?version=" + versionToDownload;
    printf("[VersionHandler] - Stage 1. Start: %s\n", url.c_str());
    HTTPUpdateResult result = Updater.update(secureClient, url.c_str(), "", [](HTTPClient *http) {
        http->addHeader("identity", config.jsonKey.c_str());
        http->addHeader("Accept", "application/octet-stream");
    });
    if( result != HTTP_UPDATE_OK ){ updateError(result); }else{
        startFileSystem();
    }
}

Debug Message

[VersionHandler] - Download task started. New version: 1.0.39

[VersionHandler] - Download task started

[VersionHandler] - Stage 1. Start: https://test.hshTest.hu/version?version=1.                                 0.39

[824832][V][HTTPClient.cpp:252] beginInternal(): url: https://test.hshTest.hu/version?version=1.0.39

[824855][D][HTTPClient.cpp:303] beginInternal(): protocol: https, host: test.hshTest.hu port:                                  443 url: /version?version=1.0.39

[826704][D][HTTPClient.cpp:598] sendRequest(): request type: 'GET' redirCount: 0



[826877][V][ssl_client.cpp:62] start_ssl_client(): Free internal heap before TLS 124311

[826885][V][ssl_client.cpp:68] start_ssl_client(): Starting socket

[826924][V][ssl_client.cpp:146] start_ssl_client(): Seeding the random number generator

[826933][V][ssl_client.cpp:155] start_ssl_client(): Setting up the SSL/TLS structure...

[826941][D][ssl_client.cpp:176] start_ssl_client(): WARNING: Skipping SSL Verification. INSECURE!

[826949][V][ssl_client.cpp:254] start_ssl_client(): Setting hostname for TLS session...

[826961][V][ssl_client.cpp:269] start_ssl_client(): Performing the SSL/TLS handshake...

[827532][V][ssl_client.cpp:290] start_ssl_client(): Verifying peer X.509 certificate...

[827540][V][ssl_client.cpp:298] start_ssl_client(): Certificate verified.

[827546][V][ssl_client.cpp:313] start_ssl_client(): Free internal heap after TLS 122551

[827555][D][HTTPClient.cpp:1170] connect():  connected to test.hshTest.hu:443

[827563][V][ssl_client.cpp:369] send_ssl_data(): Writing HTTP request with 597 bytes...

[835582][D][HTTPClient.cpp:642] sendRequest(): sendRequest code=-11



[835588][W][HTTPClient.cpp:1483] returnError(): error(-11): read Timeout

[835595][D][HTTPClient.cpp:1485] returnError(): tcp stop

[835601][V][ssl_client.cpp:321] stop_ssl_socket(): Cleaning SSL connection.

[835610][E][HTTPUpdate.cpp:234] handleUpdate(): HTTP error: read Timeout



[835617][D][HTTPClient.cpp:408] disconnect(): tcp is closed



[835622][V][ssl_client.cpp:321] stop_ssl_socket(): Cleaning SSL connection.

[VersionHandler] - Stage 1/2 error code: 0
[VersionHandler] - HTTP error: read Timeout

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[hitecSmartHome]

I'm sure the server and the url is working fine since I can download the firmware directly. The network is fast and reliable, I have tested it.

[hitecSmartHome]

Okay so i have modified the http timeout to 30000 in handleUpdate at HTTPUpdate.cpp and it started to work...

http.setTimeout(30000);

[hitecSmartHome]

Need a way to set the timeout from outside the HttpUpdate class

[me-no-dev]

HTTPUpdate Updater(30000);

[hitecSmartHome]

I get a bunch of errors when using this HTTPUpdate Updater(30000);

src/VersionHandler/VersionHandler.h:35:28: error: expected identifier before numeric constant
         HTTPUpdate Updater(15000);

[me-no-dev]

This compiles fine here. Check for issues in your code.

#include "HTTPUpdate.h"
HTTPUpdate Updater(30000);

void setup() {
  Updater.rebootOnUpdate(false);
  Updater.setFollowRedirects(HTTPC_FORCE_FOLLOW_REDIRECTS);
  Updater.onStart([]() {});
  Updater.onEnd([]() {});
  Updater.onError([](int err) {});
  Updater.onProgress([](int current, int total) {});
}

void loop() {

}

[hitecSmartHome]

Is this 2.0.17?

[me-no-dev]

3.0.1

[hitecSmartHome]

Yeah, so it does not work on 2.0.17. I got this error again even when my modification is there.

HTTPUpdateResult HTTPUpdate::handleUpdate(HTTPClient& http, const String& currentVersion, bool spiffs, HTTPUpdateRequestCB requestCB)
{

    HTTPUpdateResult ret = HTTP_UPDATE_FAILED;

    // use HTTP/1.0 for update since the update handler not support any transfer Encoding
    http.useHTTP10(true);

    http.setTimeout(60000);  // <--- ADDED THIS LINE

    http.setFollowRedirects(_followRedirects);
    http.setUserAgent("ESP32-http-Update");
    http.addHeader("Cache-Control", "no-cache");
    http.addHeader("x-ESP32-STA-MAC", WiFi.macAddress());
    http.addHeader("x-ESP32-AP-MAC", WiFi.softAPmacAddress());
    http.addHeader("x-ESP32-free-space", String(ESP.getFreeSketchSpace()));
    http.addHeader("x-ESP32-sketch-size", String(ESP.getSketchSize()));
    String sketchMD5 = ESP.getSketchMD5();
    ...

I got read timeout after 30 sec and I can't do anything about it.

[JAndrassy]

in version 2 the timeouts were no set and used correctly on WiFiClient. WiFiClient::setTimeout was hiding Stream::setTimeout used for readBytes in HTTPClient

[hitecSmartHome]

I have also set it like this

WiFiClientSecure secureClient;
secureClient.setTimeout(35);

And I have modified writeStream() in Updater.cpp

from this

/* 
Init read&timeout counters and try to read, if read failed, increase counter,
wait 100ms and try to read again. If counter > 300 (30 sec), give up/abort
*/
toRead = 0;
timeout_failures = 0;
while(!toRead) {
    toRead = data.readBytes(_buffer + _bufferLen,  bytesToRead);
    if(toRead == 0) {
        timeout_failures++;
        if (timeout_failures >= 300) {
            _abort(UPDATE_ERROR_STREAM);
            return written;
        }
        delay(100);
    }
}

to this

/* 
Init read&timeout counters and try to read, if read failed, increase counter,
wait 100ms and try to read again. If counter > 300 (30 sec), give up/abort
*/
toRead = 0;
timeout_failures = 0;
while(!toRead) {
    toRead = data.readBytes(_buffer + _bufferLen,  bytesToRead);
    if(toRead == 0) {
        timeout_failures++;
        if (timeout_failures >= 1300) {
            _abort(UPDATE_ERROR_STREAM);
            return written;
        }
        delay(100);
    }
}

Without success.

[hitecSmartHome]

I see this in Stream.cpp

int Stream::timedRead()
{
    int c;
    _startMillis = millis();
    do {
        c = read();
        if(c >= 0) {
            return c;
        }
    } while(millis() - _startMillis < _timeout);
    return -1;     // -1 indicates timeout
}

Should I modify _timeout to a constant?