Repair Backup Missing DBlock Files

[ccmgr]

Hi

I had to do a full restore on a Backup recently. Decided that now would be a god time to try from Duplicati
Turned out it had 2 dblock files missing. I effected roughly 54 files. I was able to restore same files from another source. I have 2 questions.

1. Rather then run a full repair, If I delete the versions with the missing deblocks will my next backup include the effected files

2. Is there any quick way to check if all files on remote storage exist

Ger

[drwtsn32]

Check out this thread for some info and how to go about recovering:

Missing dblocks on remote but files on local are still here

Hello, I received an email from OVH regarding the fire and my files. The report says 14 files are missing from their server (6 dblocks and 6 indexes). I ran a backup with the rebuild missing blocks flag, but it only restored the indexes not the blocks. I later check which files were on thoses missing blocks (with “affected” command) and I still have all of them on my local, I tried to run the repair database button, nothing new, and the repair command but what I get is the follow output Li…

Regarding this:

Is there any quick way to check if all files on remote storage exist

Normally Duplicati checks for the presence of all files remote files at the start of every backup operation. Has it been a long time since you ran a backup? Or did you perhaps turn off this check?

[ts678]

@ccmgr

I had to do a full restore

To clarify a little, what kind? Normal or Direct restore from backup files or something else?

@ccmgr

I effected roughly 54 files

Per The AFFECTED command, The LIST-BROKEN-FILES command or something else?
I think the commands I named need a database, so it’s a question why list wasn’t checked.
no-backend-verification Advanced option can suppress, but it’s not meant for ordinary use:

--no-backend-verification = false
If this flag is set, the local database is not compared to the remote filelist on startup. The intended usage for this option is to work correctly in cases where the filelisting is broken or unavailable.

@ccmgr

run a full repair

What’s that? The REPAIR command has two behaviors. Do you mean recreate entire database?
That’s usually the one that potentially can be painfully slow, especially if some data is hard to find.

@ccmgr

If I delete the versions with the missing deblocks will my next backup include the effected files

This is too vague. If somehow the current version wasn’t affected, the next one is likely fine now.
If files were deleted in the current version, then next backup should backup current view of them.
This isn’t special treatment. Duplicati always backs up what look to it like added or modified files.

[ccmgr]

@ts678
It was a normal restore. I used what I thought was a consistent Database.

Checked the config file and yes my bad had enabled the --no-backend-verification = false at some point and forgot to remove it. Backup in question has previously given trouble during file upload (ISP is over the Air) and I have removed versions that had failed previously due to incomplete/partial uploads.

Backup Data is mostly Daily additions with very little modification of files.

I want avoid a full database repair but my database is out of sync with the backend.

From reading the link that drwtsn32 provided my best plan of action would be purge-broken-files and run a backup ?

[ts678]

@ccmgr

my best plan of action would be purge-broken-files and run a backup ?

Recovering by purging files probably fits. You check list-broken-files before letting the purge-broken-files run. These tools appear to work best when there are missing (not corrupted) dblock (and not some other type) of files, however your situation sounds all set to go. After the purge, backup and it should backup what it needs.

[ccmgr]

Just to come back and update.

Once I purged the broken files the following backup failed to run. Got file set errors.

Unexpected difference in fileset version 328: 15/09/2020 09:11:09 (database id: 276), found 105536 entries, but expected 105545

Repair did not resolve. the issue is FilesetEntry Tables from 275 onwards contains 9 FileID’s that are not in the FileLookup Table. Once I removed these FileIDs from the FilesetEntry Table Backup now completes. All broken Files previously listed are in the latest version rather then Version 328.

This error probably goes back to a version delete when running on 2.0.4 that went amiss and the workaround at the time was to disable the file set check.

I do suspect that the backend is probably broken and I may not be able to Recover without a DB.

I plan to archive/retain this backup and with its Database and start another backup from scratch.