v14.0.0 RC2

.github/label-commenter-config.yml

@@ -91,8 +91,8 @@
       issue:
           body: |
             See [Digital Digits](https://jomjol.github.io/neural-network-digital-counter-readout) resp. [Analogue Pointers](https://jomjol.github.io/neural-network-analog-needle-readout) for an overview of all trained data.
-            If your type is not contained it can be added to our training material, see [here](https://jomjol.github.io/AI-on-the-edge-device-docs/Learn-models-with-your-own-images/).
+            If your type is not contained it can be added to our training material, see [here](https://jomjol.github.io/AI-on-the-edge-device-docs/collect-new-images/).
       discussion:
           body: |
             See [Digital Digits](https://jomjol.github.io/neural-network-digital-counter-readout) resp. [Analogue Pointers](https://jomjol.github.io/neural-network-analog-needle-readout) for an overview of all trained data.
-            If your type is not contained it can be added to our training material, see [here](https://jomjol.github.io/AI-on-the-edge-device-docs/Learn-models-with-your-own-images/).
+            If your type is not contained it can be added to our training material, see [here](https://jomjol.github.io/AI-on-the-edge-device-docs/collect-new-images/).

Changelog.md

@@ -2,57 +2,64 @@
 
 ## [Unreleased]
 
+## [14.0.0-RC1] - 2023-01-27
+
 **Stabilization and Improved User Experience**
 
 Thanks to over 80 Pull Requests from 6 contributors, we can anounce another great release with many many improvements and new features:
 
 ### Update Procedure
-Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update)
+
+Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update-ota-over-the-air)
 
 ### Changes
+
 For a full list of changes see [Full list of changes](https://github.com/jomjol/AI-on-the-edge-device/compare/v13.0.8...v14.0.0)
 
 #### Added
- - [1877](https://github.com/jomjol/AI-on-the-edge-device/pull/1877) Show WIFI signal text labels / Log RSSI value to logfile
- - Web UI caching of static files
- - Added various debug tools
- - [1798](https://github.com/jomjol/AI-on-the-edge-device/pull/1798) Add error handling for memory intensive tasks
- - [1784](https://github.com/jomjol/AI-on-the-edge-device/pull/1784) Add option to disable brownout detector
- - Added full web browser based installation mode (including initial setup of SD-card) - see [WebInstaller](https://jomjol.github.io/AI-on-the-edge-device/index.html)
- - Added [Demo Mode](https://jomjol.github.io/AI-on-the-edge-device-docs/Demo-Mode)
- - [1648](https://github.com/jomjol/AI-on-the-edge-device/pull/1648) Added trigger to start a flow by [REST](https://jomjol.github.io/AI-on-the-edge-device-docs/REST-API) API or [MQTT](https://jomjol.github.io/AI-on-the-edge-device-docs/MQTT-API/)
- - Show special images during steps `Initializing` and `Take Image` as the current camera image might be incomplete or outdated
+
+-   [1877](https://github.com/jomjol/AI-on-the-edge-device/pull/1877) Show WIFI signal text labels / Log RSSI value to logfile
+-   Web UI caching of static files
+-   Added various debug tools
+-   [1798](https://github.com/jomjol/AI-on-the-edge-device/pull/1798) Add error handling for memory intensive tasks
+-   [1784](https://github.com/jomjol/AI-on-the-edge-device/pull/1784) Add option to disable brownout detector
+-   Added full web browser based installation mode (including initial setup of SD-card) - see [WebInstaller](https://jomjol.github.io/AI-on-the-edge-device/index.html)
+-   Added [Demo Mode](https://jomjol.github.io/AI-on-the-edge-device-docs/Demo-Mode)
+-   [1648](https://github.com/jomjol/AI-on-the-edge-device/pull/1648) Added trigger to start a flow by [REST](https://jomjol.github.io/AI-on-the-edge-device-docs/REST-API) API or [MQTT](https://jomjol.github.io/AI-on-the-edge-device-docs/MQTT-API/)
+-   Show special images during steps `Initializing` and `Take Image` as the current camera image might be incomplete or outdated
 
 #### Changed
- - Migrated documentation (Wiki) to [https://jomjol.github.io/AI-on-the-edge-device-docs](https://jomjol.github.io/AI-on-the-edge-device-docs). Please help us to make it even better.
- - New OTA Update page with progress indication
- - Various memory optimizations
- - Cleanup code/Web UI
- - Updated models
- - [1809](https://github.com/jomjol/AI-on-the-edge-device/pull/1809) Store preprocessed image with ROI to RAM
- - Better log messages on some errors/issues
- - [1742](https://github.com/jomjol/AI-on-the-edge-device/pull/1742) Replace alert boxes with overlay info boxes
- - Improve log message when web UI is installed incomplete
- - [1676](https://github.com/jomjol/AI-on-the-edge-device/pull/1676) Improve NTP handling
- - HTML: improved user informations (info boxes, error hints, ...)
- - [1904](https://github.com/jomjol/AI-on-the-edge-device/pull/1904) Removed newlines in JSON and replaced all whitespaces where there was more than one
+
+-   Migrated documentation (Wiki) to <https://jomjol.github.io/AI-on-the-edge-device-docs>. Please help us to make it even better.
+-   New OTA Update page with progress indication
+-   Various memory optimizations
+-   Cleanup code/Web UI
+-   Updated models
+-   [1809](https://github.com/jomjol/AI-on-the-edge-device/pull/1809) Store preprocessed image with ROI to RAM
+-   Better log messages on some errors/issues
+-   [1742](https://github.com/jomjol/AI-on-the-edge-device/pull/1742) Replace alert boxes with overlay info boxes
+-   Improve log message when web UI is installed incomplete
+-   [1676](https://github.com/jomjol/AI-on-the-edge-device/pull/1676) Improve NTP handling
+-   HTML: improved user informations (info boxes, error hints, ...)
+-   [1904](https://github.com/jomjol/AI-on-the-edge-device/pull/1904) Removed newlines in JSON and replaced all whitespaces where there was more than one
 
 #### Fixed
- - Fixed many many things
- - [1509](https://github.com/jomjol/AI-on-the-edge-device/pull/1509) Protect `wifi.ini` from beeing deleted.
- - [1530](https://github.com/jomjol/AI-on-the-edge-device/pull/1530) Homeassistant `Problem Sensor`
- - [1518](https://github.com/jomjol/AI-on-the-edge-device/pull/1518) JSON Strings
- - [1817](https://github.com/jomjol/AI-on-the-edge-device/pull/1817) DataGraph: datafiles sorted -> newest on top
+
+-   Fixed many many things
+-   [1509](https://github.com/jomjol/AI-on-the-edge-device/pull/1509) Protect `wifi.ini` from beeing deleted.
+-   [1530](https://github.com/jomjol/AI-on-the-edge-device/pull/1530) Homeassistant `Problem Sensor`
+-   [1518](https://github.com/jomjol/AI-on-the-edge-device/pull/1518) JSON Strings
+-   [1817](https://github.com/jomjol/AI-on-the-edge-device/pull/1817) DataGraph: datafiles sorted -> newest on top
 
 #### Removed
--   n.a.
 
+-   n.a.
 
 ## [13.0.8] - 2022-12-19
 
 **Home Assistant MQTT Discovery Support**
 
-### Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update)
+### Update Procedure see [online documentation](https://jomjol.github.io/AI-on-the-edge-device-docs/Installation/#update-ota-over-the-air)
 
 ### Added
 
@@ -144,8 +151,8 @@ Improve **u**ser e**x**perience
 5.  Now you can reboot.
 
 If anything breaks you can try to
-1\. Call `http://<IP>/ota?task=update&file=firmware.bin` resp. `http://<IP>/ota?task=update&file=html.zip` if the upload successed but the extraction failed.
-1\. Use the initial_esp32_setup.zip ( <https://github.com/jomjol/AI-on-the-edge-device/wiki/Installation> ) as alternative.
+1. Call `http://<IP>/ota?task=update&file=firmware.bin` resp. `http://<IP>/ota?task=update&file=html.zip` if the upload successed but the extraction failed.
+1. Use the initial_esp32_setup.zip ( <https://github.com/jomjol/AI-on-the-edge-device/wiki/Installation> ) as alternative.
 
 ### Added
 
@@ -797,7 +804,9 @@ External Illumination
 
 -   Initial Version
 
-[Unreleased]: https://github.com/jomjol/AI-on-the-edge-device/compare/13.0.8...HEAD
+[Unreleased]: https://github.com/jomjol/AI-on-the-edge-device/compare/14.0.0-RC1...HEAD
+
+[14.0.0-RC1]: https://github.com/jomjol/AI-on-the-edge-device/compare/13.0.8...14.0.0-RC1
 
 [13.0.8]: https://github.com/jomjol/AI-on-the-edge-device/compare/12.0.1...13.0.8
 

code/components/jomjol_flowcontroll/ClassFlowInfluxDB.cpp

@@ -11,6 +11,8 @@
 #include "esp_log.h"
 #include "../../include/defines.h"
 
+#include "ClassLogFile.h"
+
 #include <time.h>
 
 static const char* TAG = "class_flow_influxDb";
@@ -108,11 +110,12 @@ bool ClassFlowInfluxDB::ReadParameter(FILE* pfile, string& aktparamgraph)
 
     if ((uri.length() > 0) && (database.length() > 0) && (measurement.length() > 0)) 
     { 
-        ESP_LOGD(TAG, "Init InfluxDB with uri: %s, measurement: %s, user: %s, password: %s", uri.c_str(), measurement.c_str(), user.c_str(), password.c_str());
+//        ESP_LOGD(TAG, "Init InfluxDB with uri: %s, measurement: %s, user: %s, password: %s", uri.c_str(), measurement.c_str(), user.c_str(), password.c_str());
+        LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Init InfluxDB with uri: " + uri + ", measurement: " + measurement + ", user: " + user + ", password: " + password);
         InfluxDBInit(uri, database, measurement, user, password); 
         InfluxDBenable = true;
     } else {
-        ESP_LOGD(TAG, "InfluxDB init skipped as we are missing some parameters");
+        LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDB init skipped as we are missing some parameters");
     }
 
     return true;

code/components/jomjol_influxdb/interface_influxdb.cpp

@@ -21,25 +21,26 @@ static esp_err_t http_event_handler(esp_http_client_event_t *evt)
     switch(evt->event_id)
     {
         case HTTP_EVENT_ERROR:
-            ESP_LOGE(TAG, "HTTP Client Error encountered");
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
             break;
         case HTTP_EVENT_ON_CONNECTED:
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Error encountered");
             ESP_LOGI(TAG, "HTTP Client Connected");
             break;
         case HTTP_EVENT_HEADERS_SENT:
-            ESP_LOGV(TAG, "HTTP Client sent all request headers");
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client sent all request headers");
             break;
         case HTTP_EVENT_ON_HEADER:
-            ESP_LOGV(TAG, "Header: key=%s, value=%s", evt->header_key, evt->header_value);
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Header: key=" + std::string(evt->header_key) + ", value="  + std::string(evt->header_value));
             break;
         case HTTP_EVENT_ON_DATA:
-            ESP_LOGV(TAG, "HTTP Client data recevied: len=%d", evt->data_len);
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client data recevied: len=" + std::to_string(evt->data_len));
             break;
         case HTTP_EVENT_ON_FINISH:
-            ESP_LOGI(TAG, "HTTP Client finished");
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client finished");
             break;
          case HTTP_EVENT_DISCONNECTED:
-            ESP_LOGI(TAG, "HTTP Client Disconnected");
+            LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP Client Disconnected");
             break;
     }
     return ESP_OK;
@@ -61,39 +62,54 @@ void InfluxDBPublish(std::string _key, std::string _content, std::string _timest
        http_config.auth_type = HTTP_AUTH_TYPE_BASIC;
     }
 
-    // generate timestamp (TODO: parse result timestamp passed as string and convert it to POSIX timestamp?)
-    time_t now = time(NULL);
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "InfluxDBPublish - Key: " + _key + ", Content: " + _content + ", Timestamp: " + _timestamp);
+
+    // Format:     #define PREVALUE_TIME_FORMAT_OUTPUT "%Y-%m-%dT%H:%M:%S%z"
+    struct tm tm;
+    strptime(_timestamp.c_str(), PREVALUE_TIME_FORMAT_OUTPUT, &tm);
+    time_t t = mktime(&tm);  // t is now your desired time_t
+
+
+//    time_t now;
+//    time(&now);
     char nowTimestamp[21];
     // pad with zeroes to get nanoseconds
-    sprintf(nowTimestamp,"%jd000000000", (intmax_t)now);
+//    sprintf(nowTimestamp,"%ld000000000", (long) now);
+    sprintf(nowTimestamp,"%ld000000000", (long) t);
 
+
+//    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "Test Time Conversion - now: " + std::to_string(now) + ", timestamp: " + std::to_string(t)  + "(correct time not used yet)");
+
     std::string payload = _influxDBMeasurement + " " + _key + "=" + _content + " " + nowTimestamp;
     payload.shrink_to_fit();
-    ESP_LOGI(TAG, "sending line to influxdb: %s\n", payload.c_str());
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "sending line to influxdb:" + payload);
+
 
     // use the default retention policy of the database
     std::string apiURI = _influxDBURI + "/api/v2/write?bucket=" + _influxDBDatabase + "/";
     apiURI.shrink_to_fit();
     http_config.url = apiURI.c_str();
-    ESP_LOGI(TAG, "API URI: %s", apiURI.c_str());
+
+    LogFile.WriteToFile(ESP_LOG_INFO, TAG, "API URI: " + apiURI);
 
     esp_http_client_handle_t http_client = esp_http_client_init(&http_config);
-    ESP_LOGI(TAG, "client is initialized%s\n", "");
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "client is initialized");
 
     esp_http_client_set_header(http_client, "Content-Type", "text/plain");
-    ESP_LOGI(TAG, "header is set%s\n", "");
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "header is set");
 
     ESP_ERROR_CHECK(esp_http_client_set_post_field(http_client, payload.c_str(), payload.length()));
-    ESP_LOGI(TAG, "post payload is set%s\n", "");
+    LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "post payload is set");
 
     esp_err_t err = ESP_ERROR_CHECK_WITHOUT_ABORT(esp_http_client_perform(http_client));
 
     if( err == ESP_OK ) {
-      ESP_LOGI(TAG, "HTTP request was performed%s\n", "");
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request was performed");
       int status_code = esp_http_client_get_status_code(http_client);
-      ESP_LOGI(TAG, "HTTP status code %d\n", status_code);
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP status code" + std::to_string(status_code));
     } else {
-      ESP_LOGW(TAG, "HTTP request failed%s\n", "");
+      LogFile.WriteToFile(ESP_LOG_DEBUG, TAG, "HTTP request failed");
     }
     esp_http_client_cleanup(http_client);
 }

docs/index.html

@@ -23,85 +23,18 @@
 
       <p>This page provides the Webinstaller and a live USB Console to your AI-on-the-edge-device.<br>
       For further information about AI-on-the-edge-device please go to <a href=https://github.com/jomjol/AI-on-the-edge-device>https://github.com/jomjol/AI-on-the-edge-device</a>.</p>
-      <p>Beside flashing the firmware you need a sd-card with configuration informations about the system and the wifi credentials.</p>
-      <p>There a two ways to make this settings:</body>p>
-      <ol>
-         <li>By hand: follow the instructions in the wiki (copy data to sd-card, setup wifi credentials by hand)</li>
-         <li>Use an integraded primitiv access point to make the first setup.<br>
-             In this case read the instructions below carefully in advance, as you have to temporaly change wifi and need a downloaded file.</li>
-      </ol>
 
       <p>Notes:</p>
       <ul>
          <li>For the installation, make sure to switch the ESP32 to Bootloader mode!</li>
-         <li>The SD-Card still must be flashed separately on a PC (See in the <a href=https://github.com/jomjol/AI-on-the-edge-device/wiki/Installation>Wiki</a>)!</li>
+         <li>Beside the firmware, also the SD-Card needs to be flashed. This can be done manually or through an initial simple access point.<br>Instructions see <a href=https://github.com/jomjol/AI-on-the-edge-device/wiki/Installation>Wiki</a>!</li>
          <li>After the installation, a manual reset might be required!</li>
          <li>Please note that not all webbrowsers and operating systems support the needed access to USB!</li>
          <li>Check the <a href=https://github.com/jomjol/AI-on-the-edge-device/wiki/Installation>Wiki</a> for additional information.</li>
       </ul>
+
       <p><esp-web-install-button manifest="manifest.json"></esp-web-install-button></p>
       <hr>
       <p style="font-size: small;">Installer and Console powered by <a href=https://esphome.github.io/esp-web-tools/ target=_blank>ESP Web Tools</a></p>
-
-
-      <h5>Using internal access point for sd-card setup</h5>
-      <p>Before starting the flash process, download the necessary file. It is a zip file, containing the initial default configuration.
-         You can identify it by the naming. It is named `AI-on-the-edge-device__remote-setup__*.zip`. Store this file locally as you will need it later. 
-      </p>
-
-      <h6>Flash the firmware with the WebInstaller</h6>
-         <div style="text-indent:50px;">
-            <p>Instructions see above.</p>
-         </div>
-
-      <h6>Connect to Device</h6>
-         <div style="text-indent:50px;">
-            <p>During the first booting, the device detects that the wifi credentials as well as the configuration informations are missing.</p>
-            <p>Therefore a simple wifi access point is initiated and a simple internal web server is startet, so the device can be setup.</p>
-            <p>The naming of the wifi is "AI-on-the-edge" and you can access it without any password.</p>
-
-            <p><img src="access-point.png" alt="Access Point" width="250"></p>
-
-            <p>You connect to the server with the fixed ip: <a href=http://192.168.4.1>http://192.168.4.1</a></p>
-         </div>
-
-      <h6>Upload initial configuration to sd-card</h6>
-         <div style="text-indent:50px;">
-            <p><img src="setup-config.png" alt="Access Point" width="300"></p>
-            <p>Use the `select file` and `upload` button to start the upload.</p>
-            <p>A warning will show up if you have choosen a possible wrong file (without default configuration).</p>
-         </div>
-
-      <h6>Upload initial configuration to sd-card</h6>
-         <div style="text-indent:50px;">
-
-            <p><img src="setup-config.png" alt="Access Point" width="300"></p>
-
-            <p>Use the `select file` and `upload` button to start the upload.<br>
-            A warning will show up if you have choosen a possible wrong file (without default configuration).</p>
-            <p>Be patient - the upload takes up to around 60s without response during this time!<br>
-            After succesfull uploading, the page will be reloaded for the next step.</p>
-         </div>
-
-      <h6>Store WLAN acces information</h6>
-         <div style="text-indent:50px;">
-            <p><img src="setup-wlan.png" alt="Access Point" width="300"></p>
-
-            <p>Here you can set your wifi credentials. Only basic settings can done here. If you need advanced features (fixed ip, ...), please use the manual setup.</p>
-            <p>Attention:</p>
-            <ul>
-               <li>Carefully check your wifi settings. To change them later on, you need to take you the sd-card and to it manually in `wlan.ini`</li>
-               <li>The informations are transfered without encryption.</li>
-            </ul>
-            <p>Finish the step by pushing `Write wlan.ini`</p>
-         </div>
-
-      <h6>Reboot</h6>
-         <div style="text-indent:50px;">
-            <p><img src="setup-reboot.png" alt="Access Point" width="300"></p>
-
-            <p>The final step is the reboot.</p>
-            <p>It will take up to 3 minutes. Afterwards you can find your device in the local network. Check you router for the IP. You can find it also in the USB Console output.</p>
-         </div>
    </body>
 </html>