How long does the re-validation in 2.5.0 take?

[Beet4]

I have the OneDrive client installed as a docker on my Unraid machine at home to sync all of my families photos to my unraid for use in PhotoPrism. A couple of days ago I updated to the 2.5.0 client version. It prompted me I needed to do a resync, I suppose because:

As such, on your first use of this client version, the database schema will be updated and the client will perform an entire re-validation of your data - both locally and online which may take an extended period of time.

So I started a resync, let it go for maybe 12-24 hours, can't really remember. New photos from my folder had updated but not from my wifes, which is a folder shared to me from her OneDrive and sits in my root OneDrive folder as a shared folder. I removed the resync parameters from the config file and rebooted but it still prompted me to do a resync. Did another run for maybe 12 hours, and rebooted without resync - still prompted me for a resync. So it has now been sitting for 48h+ in resync mode but her photos has still not synced to the local client. New photos I take now is synced to the client as expected but hers starting from the upgrade is not synced.

Can someone help me figure out what I am doing wrong, please?

Here is what the docker is currently outputting:

Performing a full scan of online data to ensure consistent local state
Fetching items from the OneDrive API for Drive ID: 4777d6625a21ea18 ........................................................................................................................................................................................................................................................................................................................................ 
Processing 43980 applicable changes and items received from Microsoft OneDrive ....... 
Performing a database consistency and integrity check on locally stored data .................... 
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Syncing changes from Microsoft OneDrive ...
Sync with Microsoft OneDrive is complete

And the same output cycles.

Here is my docker config in unraid:

Here is my config:

# Configuration for OneDrive Linux Client
# This file contains the list of supported configuration fields
# with their default values.
# All values need to be enclosed in quotes
# When changing a config option below, remove the '#' from the start of the line
# For explanations of all config options below see docs/usage.md or the man page.
#
# sync_dir = "~/OneDrive"
# skip_file = "~*|.~*|*.tmp"
# monitor_interval = "300"
# skip_dir = ""
# log_dir = "/var/log/onedrive/"
# drive_id = ""
# upload_only = "false"
# check_nomount = "false"
# check_nosync = "false"
download_only = "true"
# disable_notifications = "false"
# disable_upload_validation = "false"
# enable_logging = "false"
# force_http_11 = "false"
# local_first = "false"
# no_remote_delete = "false"
# skip_symlinks = "false"
# debug_https = "false"
# skip_dotfiles = "false"
# skip_size = "1000"
# dry_run = "false"
# min_notify_changes = "5"
# monitor_log_frequency = "6"
# monitor_fullscan_frequency = "12"
# sync_root_files = "false"
# classify_as_big_delete = "1000"
# user_agent = ""
# remove_source_files = "false"
# skip_dir_strict_match = "false"
# application_id = ""
resync = "true"
resync_auth = "true"
# bypass_data_preservation = "false"
# azure_ad_endpoint = ""
# azure_tenant_id = "common"
# sync_business_shared_items = "false"
sync_dir_permissions = "777"
sync_file_permissions = "666"
# rate_limit = "131072"
# operation_timeout = "3600"
# webhook_enabled = "false"
# webhook_public_url = ""
# webhook_listening_host = ""
# webhook_listening_port = "8888"
# webhook_expiration_interval = "600"
# webhook_renewal_interval = "300"
# webhook_retry_interval = "60"
# space_reservation = "50"
# display_running_config = "false"
# read_only_auth_scope = "false"

And finally, here is my sync_list:

/Bilder/Album/
/Bilder/Expert RAW/
/Bilder/Filmer/
/Bilder/Kompaktkamera/
/Bilder/Mobil/
/Bilder/Systemkamera/JPEG/
/Charlies Bilder/Mobil/
/Sandras Bilder/Album/
/Sandras Bilder/Mobil/

[abraunegg]

@Beet4

How long does the re-validation in 2.5.0 take?

Based on your log, it already completed long time ago.

I have the OneDrive client installed as a docker on my Unraid machine at home to sync all of my families photos to my unraid for use in PhotoPrism. A couple of days ago I updated to the 2.5.0 client version.

Please ensure you are using the 'edge' Docker tag, not the 'latest' tag.

I removed the resync parameters from the config file and rebooted but it still prompted me to do a resync. Did another run for maybe 12 hours, and rebooted without resync - still prompted me for a resync.

Every time you edit your config, if you change a parameter in the config file and/or 'sync_list' file it will prompt for a --resync.

When you get such a prompt under Docker, you should use the the Docker flag ONEDRIVE_RESYNC to avoid editing your configuration file causing a chicken and egg situation ..... please read https://github.com/abraunegg/onedrive/blob/master/docs/docker.md for more information.

So it has now been sitting for 48h+ in resync mode but her photos has still not synced to the local client. New photos I take now is synced to the client as expected but hers starting from the upgrade is not synced.

So .. this is 100% because of your 'sync_list' configuration.

The bug report here (#2832) - whilst I have been looking into this, is caused by the 'sync_list' matching mechanism. This will be your issue because of these entries:

/Bilder/Album/
/Bilder/Expert RAW/
/Bilder/Filmer/
/Bilder/Kompaktkamera/
/Bilder/Mobil/
/Bilder/Systemkamera/JPEG/
/Charlies Bilder/Mobil/
/Sandras Bilder/Album/
/Sandras Bilder/Mobil/

If you change this to:

/Bilder/Album/*
/Bilder/Expert RAW/*
/Bilder/Filmer/*
/Bilder/Kompaktkamera/*
/Bilder/Mobil/*
/Bilder/Systemkamera/JPEG/*
/Charlies Bilder/Mobil/*
/Sandras Bilder/Album/*
/Sandras Bilder/Mobil/*

This most likely resolve your current issue in the files not downloading.

Actions for you:

1. Use the 'edge' Docker tag
2. Edit your config and stop using those resync values in your config file
3. Edit your 'sync_list'
4. Restart your Docker container using the ONEDRIVE_RESYNC flag (that is what it is there for)

[Beet4]

Wow, thanks for the super quick answer and support! I'll test your suggestions, run a resync with ONEDRIVE_RESYNC instead, and get back with the results.

[Beet4]

So I've followed the steps and the client crashed and docker was stopped, see log below. Using edge.

useradd warning: onedrive's uid 99 outside of the UID_MIN 1000 and UID_MAX 60000 range.
# Running container as user: onedrive
# Base Args: --monitor
# We are performing a --resync
# Adding --resync --resync-auth
# Changing ownership permissions on /onedrive/data and /onedrive/conf to onedrive:users
# Launching 'onedrive' as onedrive via gosu
Reading configuration file: /onedrive/conf/config
Configuration file successfully loaded
Deleting the saved application sync status ...
Using IPv4 and IPv6 (if configured) for all network operations
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Configuring Global Azure AD Endpoints

WARNING: Application has been configured to cleanup local files that are not present online.
WARNING: Local data loss MAY occur in this scenario if you are expecting data to remain archived locally.

OneDrive synchronisation interval (seconds): 300
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Fetching items from the OneDrive API for Drive ID: 4777d6625a21ea18 ........................................................................................................................................................................................................................................................................................................................................ 
Finished processing self generated /delta JSON response from the OneDrive API
Processing 42731 applicable changes and items received from Microsoft OneDrive ............................................. 
Syncing this OneDrive Personal Shared Folder: Charlies Bilder
Fetching items from the OneDrive API for Drive ID: d83c2d89e64b6e85 ...Attempting to perform a database vacuum to optimise database
Database vacuum is complete
std.json.JSONException@std/json.d(641): Key not found: id
----------------
/usr/lib/ldc/x86_64-linux-gnu/include/d/std/exception.d:519 pure @safe noreturn std.exception.bailOut!(std.json.JSONException).bailOut(immutable(char)[], ulong, scope const(char)[]) [0x5584f3c74763]
??:? inout pure ref @safe inout(std.json.JSONValue) std.json.JSONValue.opIndex(return scope immutable(char)[]) [0x14670ba97dc9]
src/sync.d:3969 [0x5584f3aa5ae5]
src/sync.d:4009 [0x5584f3aa5de0]
src/sync.d:4009 [0x5584f3aa5de0]
src/sync.d:4009 [0x5584f3aa5de0]
src/sync.d:4009 [0x5584f3aa5de0]
src/sync.d:3925 [0x5584f3a93070]
src/sync.d:6870 [0x5584f3ab8787]
src/sync.d:6883 [0x5584f3ab8b46]
src/sync.d:6883 [0x5584f3ab8b46]
src/sync.d:6776 [0x5584f3a8bf3a]
src/sync.d:1098 [0x5584f3a82067]
src/sync.d:598 [0x5584f3a8000c]
src/main.d:1225 [0x5584f3b3fd36]
src/main.d:1014 [0x5584f3b3d6c6]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x14670b79db7b]
??:? _d_run_main2 [0x14670b79d995]
??:? _d_run_main [0x14670b79d7ed]
/usr/lib/ldc/x86_64-linux-gnu/include/d/core/internal/entrypoint.d:42 [0x5584f3b701e1]
??:? [0x14670b41c249]
??:? __libc_start_main [0x14670b41c304]
??:? [0x5584f3a49100]

** Press ANY KEY to close this window ** 

[abraunegg]

That issue has not been seen in over 9 months testing prior to release on Monday.

Please raise a bug report.

[Beet4]

I've raised a bug report here: #2835

I also tried :latest with the same issue.