I have two backups setup. One that backs up all my Photos and Videos, and one that backs up all my music. The former is abour 160 GB and has “20 versions” and is encrypted, while the latter is only 80 GB with “2 versions”. The latter is also not encrypted, as it’s all just MP3s, OGGs, WAVs, etc, so nothing private. However, the processing of the latter (the music one) takes just over 10 minutes to process every day, even though nothing’s changed and nothing gets uploaded. Running the “compare” command show files added a couple of months ago. For comparison, the bigger photos backup processes in about 1.5 minutes.
Does anyone have any ideas why the music backup would be so slow? They’re both on the same hard drive, and both backing up to B2 storage…
Maybe it’s this problem: Backup not saved if only metadata has changed · Issue #4312 · duplicati/duplicati · GitHub
(Note that the title of this issue is a bit incorrect, IMO.)
If Duplicati thinks a timestamp doesn’t match, it will re-process a file which takes time. If the contents of the file and its metadata have not actually changed, nothing is uploaded on the back end. If this is true for ALL files in the backup set, then the backup snapshot version is also discarded (by default). The next backup repeats the process.
You can confirm this is the issue by watching About → Show Log → Live → Verbose (while the backup is running). You may see entries like this:
Checking file for changes <redacted>, new: False, timestamp changed: True, size changed: False, metadatachanged: False, 9/4/2020 1:07:47 PM vs 9/4/2020 1:07:47 PM
You can probably solve this by setting --upload-unchanged-backups=true temporarily. Run a backup then then you can clear this option.
2 Likes
Yes, that definitely seems to have been the problem. After running a backup with the --upload-unchanged-backups advanced option, the compare command line shows the changes (none) of that last backup, instead of just showing the last one with changes, and a subsequent backup took 42 seconds (and after turning the advanced option off again, it took only 39 seconds :P). Thank you!
1 Like
Great! Glad that solved your issue.