Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

InfluxDB not working in 14.0.0-RC1 #1933

Closed
jhandke opened this issue Jan 28, 2023 · 13 comments
Closed

InfluxDB not working in 14.0.0-RC1 #1933

jhandke opened this issue Jan 28, 2023 · 13 comments
Assignees
@jomjol
Labels
bug Something isn't working

Comments

@jhandke
Copy link

jhandke commented Jan 28, 2023

The Problem

Nach dem Update eines meiner Geräte auf 14.0.0-RC1 funktioniert das Senden der Werte zur InfluxDB nicht mehr. Im Log steht, dass der Status "Sending InfluxDB" per MQTT geschickt wird, in der Datenbank kommt allerdings nichts an.

Version

14.0.0-RC1

Logfile

[0d00h00m04s] 2023-01-28T22:35:45 <INF> [SNTP] TimeServer: 192.168.178.1
[0d00h00m04s] 2023-01-28T22:35:45 <INF> [SNTP] Configuring NTP Client...
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [SNTP] Time zone set to CEST-1CET
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [SNTP] Time is already set: 2023-01-28 23:35:46
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [MAIN] =================================================
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [MAIN] ==================== Startup ====================
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [MAIN] =================================================
[0d00h00m04s] 2023-01-28T23:35:46 <INF> [MAIN] Release: v14.0.0-RC1 (Commit: 245302c+), Date/Time: 2023-01-27 20:30, Web UI: Release: v14.0.0-RC1 (Commit: 245302c+)
[0d00h00m05s] 2023-01-28T23:35:46 <INF> [MAIN] Reset reason: Via esp_restart
[0d00h00m05s] 2023-01-28T23:35:46 <INF> [MAIN] WLAN-Settings - RSSI-Threashold: 0
[0d00h00m10s] 2023-01-28T23:35:51 <INF> [MAIN] =================================================
[0d00h00m10s] 2023-01-28T23:35:51 <INF> [MAIN] ================== Main Started =================
[0d00h00m10s] 2023-01-28T23:35:51 <INF> [MAIN] =================================================
[0d00h00m10s] 2023-01-28T23:35:51 <INF> [MAIN] The device has 8 MBytes of PSRAM
[0d00h00m12s] 2023-01-28T23:35:53 <INF> [MAIN] Initialization completed successfully!
[0d00h00m14s] 2023-01-28T23:35:55 <DBG> [MQTT] Digitizer interval is 3.0 minutes => setting MQTT LWT timeout to 7.5 minutes.
[0d00h00m14s] 2023-01-28T23:35:55 <DBG> [MQTT IF] URI: mqtt://192.168.178.12:1883, clientname: watermeter, user: watermeter, password: XXXXXXXX, maintopic: meter/water, last-will-topic: meter/water/connection, keepAlive: 450, RetainFlag: 0
[0d00h00m14s] 2023-01-28T23:35:55 <INF> [MQTT IF] Init
[0d00h00m14s] 2023-01-28T23:35:55 <INF> [MQTT IF] Client started, waiting for established connection...
[0d00h00m14s] 2023-01-28T23:35:55 <DBG> [TFLITE SERVER] ----------------------------------------------------------------
[0d00h00m14s] 2023-01-28T23:35:55 <INF> [TFLITE SERVER] Round #1 started
[0d00h00m14s] 2023-01-28T23:35:55 <DBG> [MQTT IF] Publish skipped. Client not initalized or not connected. (topic: meter/water/status)
[0d00h00m15s] 2023-01-28T23:35:57 <DBG> [MAIN SERVER] info_get_handler
[0d00h00m21s] 2023-01-28T23:36:03 <DBG> [MAIN SERVER] info_get_handler
[0d00h00m22s] 2023-01-28T23:36:03 <DBG> [MAIN SERVER] info_get_handler
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT SERVER] Publishing static MQTT topics...
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT IF] Published topic: meter/water/MAC, content: (removed) (msg_id=3)
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT IF] Published topic: meter/water/IP, content: 192.168.178.21 (msg_id=4)
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT IF] Published topic: meter/water/hostname, content: watermeter (msg_id=5)
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT IF] Published topic: meter/water/interval, content: 3.0 (msg_id=6)
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT SERVER] Publishing system MQTT topics...
[0d00h00m24s] 2023-01-28T23:36:05 <DBG> [MQTT IF] Published topic: meter/water/uptime, content: 24 (msg_id=7)
[0d00h00m24s] 2023-01-28T23:36:06 <DBG> [MQTT IF] Published topic: meter/water/freeMem, content: 2966054 (msg_id=8)
[0d00h00m24s] 2023-01-28T23:36:06 <DBG> [MQTT IF] Published topic: meter/water/wifiRSSI, content: -58 (msg_id=9)
[0d00h00m24s] 2023-01-28T23:36:06 <DBG> [MQTT IF] Published topic: meter/water/CPUtemp, content: 36 (msg_id=10)
[0d00h00m24s] 2023-01-28T23:36:06 <DBG> [MQTT IF] Published topic: meter/water/status, content: Aligning (msg_id=11)
[0d00h00m36s] 2023-01-28T23:36:31 <INF> [SNTP] Time is now successfully synced with NTP Server 192.168.178.1: 2023-01-28 23:36:31
[0d00h00m43s] 2023-01-28T23:36:37 <DBG> [MQTT IF] Published topic: meter/water/status, content: Digitalization of ROIs (msg_id=12)
[0d00h00m43s] 2023-01-28T23:36:37 <DBG> [CNN] doFlow after alignment
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [TFLITE] CTfLiteClass::LoadModel
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard/config/dig-class11_1411_s2_q.tflite
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [TFLITE] CTfLiteClass::MakeAllocate
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [CNN] Processing Number 'main'
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [CNN] ROI #0 - TfLite
[0d00h00m44s] 2023-01-28T23:36:38 <DBG> [CNN] CNN Type: Digital
[0d00h00m44s] 2023-01-28T23:36:39 <DBG> [CNN] ROI #1 - TfLite
[0d00h00m44s] 2023-01-28T23:36:39 <DBG> [CNN] CNN Type: Digital
[0d00h00m45s] 2023-01-28T23:36:39 <DBG> [CNN] ROI #2 - TfLite
[0d00h00m45s] 2023-01-28T23:36:39 <DBG> [CNN] CNN Type: Digital
[0d00h00m45s] 2023-01-28T23:36:40 <DBG> [CNN] ROI #3 - TfLite
[0d00h00m45s] 2023-01-28T23:36:40 <DBG> [CNN] CNN Type: Digital
[0d00h00m46s] 2023-01-28T23:36:40 <DBG> [CNN] ROI #4 - TfLite
[0d00h00m46s] 2023-01-28T23:36:40 <DBG> [CNN] CNN Type: Digital
[0d00h00m46s] 2023-01-28T23:36:40 <DBG> [MQTT IF] Published topic: meter/water/status, content: Digitalization of ROIs (msg_id=13)
[0d00h00m46s] 2023-01-28T23:36:41 <DBG> [CNN] doFlow after alignment
[0d00h00m46s] 2023-01-28T23:36:41 <DBG> [TFLITE] CTfLiteClass::LoadModel
[0d00h00m46s] 2023-01-28T23:36:41 <DBG> [TFLITE] CTfLiteClass::ReadFileToModel: /sdcard/config/ana-cont_11.3.0_s2.tflite
[0d00h00m47s] 2023-01-28T23:36:41 <DBG> [TFLITE] CTfLiteClass::MakeAllocate
[0d00h00m47s] 2023-01-28T23:36:41 <DBG> [CNN] Processing Number 'main'
[0d00h00m47s] 2023-01-28T23:36:41 <DBG> [CNN] ROI #0 - TfLite
[0d00h00m47s] 2023-01-28T23:36:41 <DBG> [CNN] CNN Type: Analogue
[0d00h00m47s] 2023-01-28T23:36:42 <DBG> [CNN] After Invoke
[0d00h00m47s] 2023-01-28T23:36:42 <DBG> [CNN] ROI #1 - TfLite
[0d00h00m47s] 2023-01-28T23:36:42 <DBG> [CNN] CNN Type: Analogue
[0d00h00m48s] 2023-01-28T23:36:42 <DBG> [CNN] After Invoke
[0d00h00m48s] 2023-01-28T23:36:42 <DBG> [CNN] ROI #2 - TfLite
[0d00h00m48s] 2023-01-28T23:36:42 <DBG> [CNN] CNN Type: Analogue
[0d00h00m48s] 2023-01-28T23:36:43 <DBG> [CNN] After Invoke
[0d00h00m48s] 2023-01-28T23:36:43 <DBG> [CNN] ROI #3 - TfLite
[0d00h00m48s] 2023-01-28T23:36:43 <DBG> [CNN] CNN Type: Analogue
[0d00h00m49s] 2023-01-28T23:36:43 <DBG> [CNN] After Invoke
[0d00h00m49s] 2023-01-28T23:36:43 <DBG> [MQTT IF] Published topic: meter/water/status, content: Post-Processing (msg_id=14)
[0d00h00m49s] 2023-01-28T23:36:44 <DBG> [CNN] getReadout _analog=0, _extendedResolution=0, prev=-1
[0d00h00m49s] 2023-01-28T23:36:44 <DBG> [CNN] PointerEvalAnalogNew - No predecessor - Result = 8 number: 8.091048 numeral_preceder = -1 Analog_error = 3
[0d00h00m49s] 2023-01-28T23:36:44 <DBG> [CNN] PointerEvalAnalogNew - number unambiguous, no correction necessary - result = 2 number: 2.508388 numeral_preceder = 8 Analog_error = 3
[0d00h00m49s] 2023-01-28T23:36:44 <DBG> [CNN] PointerEvalAnalogNew - number ambiguous, correction upwards - result = 3 number: 3.138093 numeral_preceder = 2 Analog_error = 3
[0d00h00m50s] 2023-01-28T23:36:44 <DBG> [CNN] PointerEvalAnalogNew - number ambiguous, correction upwards - result = 8 number: 8.239669 numeral_preceder = 3 Analog_error = 3
[0d00h00m50s] 2023-01-28T23:36:44 <DBG> [CNN] getReadout _analog=0, _extendedResolution=0, prev=8
[0d00h00m50s] 2023-01-28T23:36:44 <INF> [POSTPROC] main: Raw: 00518.8328, Value: 518.8328, Status: no error
[0d00h00m51s] 2023-01-28T23:36:45 <DBG> [MQTT IF] Published topic: meter/water/status, content: Sending MQTT (msg_id=15)
[0d00h00m51s] 2023-01-28T23:36:45 <DBG> [MQTT SERVER] Publishing system MQTT topics...
[0d00h00m51s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/uptime, content: 51 (msg_id=16)
[0d00h00m51s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/freeMem, content: 2968034 (msg_id=17)
[0d00h00m51s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/wifiRSSI, content: -60 (msg_id=18)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/CPUtemp, content: 37 (msg_id=19)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT] Publishing MQTT topics...
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/value, content: 518.8328 (msg_id=20)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/error, content: no error (msg_id=21)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/rate, content: 0.000085 (msg_id=22)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/rate_per_time_unit, content: 0.000085 (msg_id=23)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/changeabsolut, content: 0.0001 (msg_id=24)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/rate_per_digitalization_round, content: 0.0001 (msg_id=25)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/raw, content: 00518.8328 (msg_id=26)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/timestamp, content: 2023-01-28T23:36:01+0100 (msg_id=27)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/main/json, content: { "value": "518.8328", "raw": "00518.8328", "pre": "518.8328", .. (msg_id=28)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/status, content: Sending InfluxDB (msg_id=29)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [MQTT IF] Published topic: meter/water/status, content: Flow finished (msg_id=30)
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [TFLITE SERVER] CPU Temperature: 37°C
[0d00h00m52s] 2023-01-28T23:36:46 <DBG> [TFLITE SERVER] WIFI Signal (RSSI): -59dBm
[0d00h00m52s] 2023-01-28T23:36:46 <INF> [TFLITE SERVER] Round #1 completed (38 seconds)
[0d00h01m33s] 2023-01-28T23:37:27 <DBG> [MAIN SERVER] info_get_handler
[0d00h01m39s] 2023-01-28T23:37:33 <DBG> [OTA FILE] log_get_last_part_handler

Expected Behavior

No response

Screenshots

No response

Additional Context

No response
@jhandke jhandke added the bug Something isn't working label Jan 28, 2023
@jomjol
Copy link
Owner

jomjol commented Jan 29, 2023

Ich kann den Bug bestätigen. Welche Version genau hast du den vorher verwendet?

@jhandke
Copy link
Author

jhandke commented Jan 29, 2023

Vorher war Version 13.0.8 installiert, glaube ich.

@jomjol
Copy link
Owner

jomjol commented Jan 29, 2023

Okay - das ist merkwürdig, denn dort haben wir nichts geändert. Ich vermute, dass ich den Fehler gefunden habe, der wäre aber schon viel länger drin.

@jomjol jomjol mentioned this issue Jan 29, 2023
Merged
@jomjol
Copy link
Owner

jomjol commented Jan 29, 2023

Teste bitte mal: https://github.com/jomjol/AI-on-the-edge-device/releases/tag/v14.0.0-RC2

@jhandke
Copy link
Author

jhandke commented Jan 29, 2023 •
edited
Loading

Die Messwerte kommen wieder an. Vielen Dank!
Kann es sein, dass jetzt der Timestamp für Influx um eine oder zwei Stunden verschoben ist?
Im Log steht
[0d00h15m49s] 2023-01-29T17:27:52 <INF> [INFLUXDB] sending line to influxdb:gasmeter main/value=18157.925 1675006044000000000
was für Sunday, 29. January 2023 15:27:24 GMT steht.

Edit: im Log steht auch, dass der Header korrekt gesetzt wurde:
[0d00h15m49s] 2023-01-29T17:27:52 <DBG> [INFLUXDB] Header: key=Date, value=Sun, 29 Jan 2023 16:27:52 GMT

@jhandke
Copy link
Author

jhandke commented Jan 29, 2023

Bei beiden Geräten mit Version 14.0.0-RC2 oder -RC7 tritt noch ein Fehler auf:
[0d00h12m52s] 2023-01-29T23:52:12 <INF> [INFLUXDB] sending line to influxdb:watermeter main/value=519.3148 1675029102000000000
Der Timestamp für Influx sollte in UTC sein. Hier entspricht er Sunday, 29. January 2023 21:51:42.
Vielleicht wird die korrekte Zeitzone erst nach der NTP Synchronisation gesetzt und vorher schon Werte an Influx gesendet.
Andernfalls kann der Timestamp auch weggelassen werden, InfluxDB nimmt dann den Zeitpunkt des Eintreffens am Server, siehe auch die Dokumentation.

@caco3 caco3 removed the 14.0.0-RC2 label Jan 30, 2023
@jomjol
Copy link
Owner

jomjol commented Jan 31, 2023

Danke für den Input. Ich habe in der Tat am Timestamp etwas geändert und ihne auf den Timestamp der Bildaufnahme gesetzt, dabei aber nicht UTC-Zeit berücksichtigt.
Da muss ich wohl nochmal ran, wobei am einfachsten vermutlich das Weglassen des Timestamps ist.

@caco3
Copy link
Collaborator

caco3 commented Feb 2, 2023

ja, lass den TS weg. es kann manchmal sein, dass NTP sync nicht von anfang an da ist.
zudem gibt es anwender, welche explizit kein NTP verwenden wollen

@jomjol
Copy link
Owner

jomjol commented Feb 2, 2023

Ich habe jetzt die korrekte Umrechnung in UTC gemacht - ist die sauberere Lösung

@jomjol
Copy link
Owner

jomjol commented Feb 2, 2023

Du kannst jetzt mal die neueste Rolling oder Master probieren (noch kein Release), dort ist jetzt die Umrechung auf UTC drin.

@jhandke
Copy link
Author

jhandke commented Feb 26, 2023

In Version 15.0.1 tritt der Fehler immer noch auf:

[0d00h00m58s] 2023-02-26T14:20:39 <INF> [INFLUXDB] sending line to influxdb:gasmeter main/value=18449.675 1677410412000000000

Der Timestamp entspricht Sun Feb 26 2023 11:20:12 GMT+0000, wobei die aktuelle UTC Zeit 13:20:12 Uhr ist.

@jhandke
Copy link
Author

jhandke commented Feb 26, 2023

Ich denke dass das Problem hier ist, dass strptime() das %z im Datumsformat nicht verarbeitet und daher auch kein Offsetwert im tm Struct steht, welcher dann von gmtime() verwendet wird:
espressif/esp-idf#2219 (comment)

Ich habe mal ein Logging für time_t t in InfluxDBPublish im 15.0.1 Tag eingebaut und erhalte folgendes:
[0d03h00m51s] 2023-02-26T20:13:02 <DBG> [INFLUXDB] time_t t = 1677435153
Der Timestamp entspricht Sun Feb 26 2023 18:12:33 GMT+0000, also genau mit genau der Stunde Offset, die strptime() hier verliert.

jomjol added a commit that referenced this issue Feb 27, 2023
@jomjol
Bugfix #1933 (again :-))
9f2e91a
@jomjol
Copy link
Owner

jomjol commented Feb 28, 2023

Repaired in the newest rolling and 15.0.2

@jomjol jomjol closed this as completed Feb 28, 2023
jomjol added a commit that referenced this issue Mar 1, 2023
@caco3 @jomjol
Master (#2104)
8c7e86f 
* Replace deprecated actions (#2016)

* Update build.yaml

* Update build.yaml

* Update manual-update-webinstaller.yml

* Update manual-update-webinstaller.yml

* Update manual-update-webinstaller.yml

* Update manual-update-webinstaller.yml

* Update build.yaml

* preparations for v15.0 (#2063)

* Migrated parameters, see #2023

* remove no longer used "topic" parameter. This is a backport from b21e3c6

* Fix wrong url-encoding, see #2036 resp. #2036

* Threashold -> Threshold

* updated changelog

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>

* DataLogActive is true by default

* updated changelog

* Bugfix #1933 (again :-))

* Update Influxdb

* re-add missing dropdownbox filling for Postprocessing Individual Parameters

* stop auto filling the release notes, it causes more confusion than it helps

* Update Changelog.md

---------

Co-authored-by: CaCO3 <caco@ruinelli.ch>
Co-authored-by: jomjol <30766535+jomjol@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jomjol jomjol

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@caco3 @jhandke @jomjol