Just to be clear - if you are restoring from a backup without a database, Duplicati will use the dlist files to generate the list of what’s in the backup and can be restored.
Once you choose what you want to restore, Duplicati will then make a temporary database of what’s needed to restore the selected files. This means if you choose to restore everything, then Duplicati will be creating an “everything” temp database.
So the “big time saver” comment may ore may not apply depending on what you’re restoring.
One of these days I may do a test of a database recreate vs. restore all direct from backup just to see if there’s much of a time difference. I suspect there will be as the recreate has to rebuild the entire history of every file while the restore only needs the database for one version.
I’d like to reopen this thread because I’ve been working with ts678 on a very similar issue. I didn’t have bad memory, but I did experience raid corruption due to overheating and ended up with several corrupted files in several backup directories. Much thanks to ts678 for his dogged persistence in my daily mutterings, but in a nutshell it comes down to this: Why should a hash mismatch not be fixable/replaceable? No matter what, I could not get past this seemingly easy problem and in most cases had to move the entire backup to cold storage (just in case) and start over. list-broken and purge-broken stubbornly refused to find anything wrong and in many cases backups continued to work. But if I ever did a full-verification pass, the bad file would rear its ugly head, which of course meant that whatever files were backed up in that block were unretrievable.
As far as ‘moving the bad hash file aside’ (as commented above), ts678 made several suggestions, but none were successful in moving the file and getting duplicati to rebuild the block. Even deleting and recreating the DB didn’t help. In one instance I tried deleting the fileset that contained the bad dblock file and that threw even more errors at me during the next backup about wrong number of filesets and/or volumes.
In an effort to enable backup verification from my server, I turned on the upload-verification-file and used the python script in the duplicati utility directories. I don’t know how long finding and comparing two hashes is supposed to take, but even on relatively small directories (500 duplicati files and 12GB) the python script took over 2 hours! Some larger backup sets took over 24 hours, which made it an impractical tool for my use. Perhaps someone with a lot more experience in python, databases, and json files could help me understand what is going on.
I thought about submitting a bug report, but short of sending one of my trashed backup files directories, I had no idea how developers would duplicate the issue - other than creating a small backup on a raid, then purposely trashing the raid while a write was happening. I suppose you could take a dblock file from a successful small backup and hexedit it to mess with the hash - that might be a way to induce the error and then figure out how to fix it. Just a thought.
I’m running ubuntu 18.04 and snap nextcloud 20.0.6, using webdav in the duplicati destination.
