Local backup of solectrus

[Timokirch]

Hi all,

I fooled around with my docker host and I wonder if there is a good strategy to backup solectrus locally? Maybe because I am an absolut newbie I wonder what I would need to backup. For sure my docker-compose and .env are save (a simple copy to my notion and versioning in portainer) and my docker host is getting exported once a day to a NAS with cloud sync. For me it was not enough since I wanted a backup of the data & settings (like electricity prices). I was able to successfully backup influx once a day via cron (docker exec -it solectrus-influxdb-1 influx backup /backup/$(date '+%Y-%m-%d')/ -t *yourtokengoeshere*). For this I simply added - /backup/solectrus/influx:/backup to the volumes section of influxdb. As I tried around further I deleted the container and the volume of influx, recreated and wondered that all data & settings of solectrus are still there (without a restore).

So I wonder how and what are you backing up and where solectrus is storing its data & settings :)

Thanks for sharing and explaining (remember I have no experience with influx and whatever technology is behind solectrus).

Kind regards

[ledermann]

Great that you have found a way to back up InfluxDB with the CLI. This is the right way to create a local backup of your measurement data.

As I tried around further I deleted the container and the volume of influx, recreated and wondered that all data & settings of solectrus are still there (without a restore).

This seems to be caused by caching. Every InfluxDB query is cached in Redis for performance reasons. So, when you delete the InfluxDB volume, the cache results are still present.

If you change data in InfluxDB in an unusual way (e.g. deleting the volume, importing historical data, etc.), you must then clear the Redis cache by running this command:

docker exec -it solectrus-redis-1 redis-cli FLUSHALL

BTW: For backing up InfluxDB to Amazon S3, you can use this tool:
https://github.com/solectrus/influxdb2-s3-backup

For PostgreSQL (electricity prices and settings) there is a similar tool:
https://github.com/eeshugerman/postgres-backup-s3

I use both to back up the live demo once a day.

[Timokirch]

Thank you very much for your input I was able to create local backups and restore everything after a fresh setup. Here are my steps taken.

I added these two to the docker-compose.yml

influxdb volume: - /backup/solectrus/influx:/backup

db volume: - /backup/solectrus/postgres:/backup

Backup scripts (via host):

influx: docker exec -it solectrus-influxdb-1 influx backup /backup/$(date '+%Y-%m-%d')/ -t *yourtokengoeshere*

postgres: docker exec -it solectrus-db-1 pg_dump -h localhost -U postgres -c -f /backup/$(date '+%Y-%m-%d').dump solectrus_production -Fc

Restore (on a fresh installbase)

shutdown all container besides influx, postgres and redis

docker exec -it solectrus-influxdb-1 influx bucket delete -n solectrus -t *yourtokengoeshere*
docker exec -it solectrus-influxdb-1 influx restore /backup/*DATEFOLDER*
docker exec -it solectrus-db-1 dropdb -h localhost -U postgres -f solectrus_production
docker exec -it solectrus-db-1 createdb -h localhost -U postgres solectrus_production
docker exec -it solectrus-db-1 pg_restore -h localhost -U postgres -d solectrus_production -C /backup/2024-04-14.dump
docker exec -it solectrus-redis-1 redis-cli FLUSHALL

run the pg_restore command twice!

If you are recreating your stack run rm -rf /volume1/docker/solectrus/(or wherever your volumes are) after deleting your old stack and before your up command.

Start all other containers and enjoy your new but old but still new instance ;)

[ledermann]

Great, thanks a lot for your write-up! 👍

[weinberg64]

Hallo,
i have followed your steps above, but when executing the following error message appears:

pi@raspberrypi:~ $ docker exec -it solectrus-influxdb-1 influx backup /backup/$(date '+%Y-%m-%d')/ -t yourtokengoeshere
2024/08/14 22:47:06 INFO: Downloading metadata snapshot
Error: failed to backup metadata: failed to download metadata snapshot: 401 Unauthorized: unauthorized access

What am I doing wrong?
Kind regards

[ledermann]

The token seems to be wrong. Did you really replace yourtokengoeshere with your own token? An admin token with full rights is required to perform backups.