Welcome to the forum @landgarden
Nice writeup. One thing I’d like to know is what Storage Type is used for the Destination, and whether Destination has enough transfer capability to consider verification at Duplicati by The TEST command.
Looks to me like a bug introduced in 2015 when record of deleted files was saved for 2 additional hours:
Added grace-period for incomplete uploads to avoid backups stopping sue to the Apache WebDAV issues.
That’s probably (not tested) how deleted files got in duplicati-verification.json as State 5 to make issues.
DuplicatiVerify.ps1 and false warnings #2989 was the PowerShell report, but fix was only in PowerShell.
is possibly another place it could have been fixed, but maybe somebody thought that was a worse idea.
Regardless, the only place you can get a fast correction is in the Python script. Do you code in Python?
filename = file["Name"]
hash = file["Hash"]
size = file["Size"]
+ state = file["State"]
fullpath = os.path.join(folder, filename)
if not os.path.exists(fullpath):
- print "File missing: ", fullpath
- errorCount += 1
+ if state != 5:
+ print "File missing: ", fullpath
+ errorCount += 1
else:
checked += 1
print "Verifying file ", filename
experimentally seems to fix the issue, but I’m not a Python programmer, so if any are, correct if needed.
Actually, if any Python programmer would like to convert the script for Python 3, that would also be nice.
There is a fixed-in-Canary-but-not-Beta bug leaving old data at end of verification file if new data is less, however I think this can only occur when Duplicati updates the file directly (Type is Local folder or drive).
If you have a fast and free connection to the destination, you might alternatively test all the file content in The TEST command, specifying all for the sample count. That will catch any files that got corrupted…
Regardless, could you file an Issue on this (citing this topic), in the hope that a developer will work on it?