What happens when a backup is deleted manually?

1

Hi all,

I am trying to get my backups offsite without having to use a cloud storage option. I am looking at the possibility of using something like resilio sync to sync my backup file to an offsite hard drive. The only risk I can see in something like this is if one or all the files get deleted at the offsite location, and this deltes all or some of the files in my local folder.

Will Duplicati know that some files are missing and start a new backup to fix the trouble, or will all be lost?

2

Duplicati will know some files are missing. If they are dindex or dlist files then Duplicati will just recreate them, but if they are dblock files it won’t start any new backups until the issue is resolved (or you tell Duplicati to ignore the problem).

It works this way because of Duplicati’s versioning process. Since new backups only upload what’s changed since the last backup Duplicati relies on the old backups for the unchanged parts of the files. If an old backup is corrupted / missing in some way then uploading a new backup “on top” of it wouldn’t do you any good because the new backup might not be restorable since the old backup (containing some of the data needed for restore) isn’t viable.

If a dblock file is missing or corrupt then some backup data has been lost and there’s little Duplicati can do about it other than to pretend the corrupted versions no longer exist and continue on.

Note that Duplicati CAN restore from a backup even with missing dblock files - it will simply restore whatever it can and fill in the “missing” parts of the restored files with 0. Of course for some file types they’ll be unusable, but for others useful information could still be recovered.

By the way - that was a great question!

If you want to see what it’s like when that happens I’d suggest creating a test backup (just a few files to a local folder), delete one of the Duplicati files and see what happens. :slight_smile:

3

I’d like to add that while it’s true that dindex files are recreated, dlist files can’t be recreated as such. If one of the dlist files are lost, then that “snapshot” is no longer available. Of course if you have many snapshots this may be a big deal to you. You can still back up and restore from any other point in time where you backed up. But the one backup associated with that dlist file is no longer available.

4

Really? I could have sworn I tested a missing dlist file scenario and a database Repair rebuild it (as I assume a Recreate would).

But I may be mis-remembering - I’ll try to remember to retest that situation at some point.

Edit:
OK - I tested (see below) and can state that with 2.0.3.9 a missing dlist file:

  1. Will stop backups from being run
  2. Will NOT stop the version from being listed in a Job based restore but WILL stop it from being listed in a “Direct” restore
  3. Will cause a restore of that version to FAIL with a “Found1 files that are missing” error unless no-backend-verification=true is in the job config
  4. Can be regenerated with a database Repair

So it appears the same local database data used to recreate a dlist file can also be used to supplement a restore as long as we can get past the backend-verification step.

I tested a missing dlist failure scenario as follows:

  1. Moved dlist file to another folder
  2. Ran backup (failed)
  3. Got “Error” window linking to Job log, but “Found 1 files that are missing from the remote storage, please run repair” message was in global log
  4. Ran database “Repair” step
  5. Confirmed “missing” dlist file had been recreated (name matched original file that had been moved to another folder, but new file had current date)
  6. Ran backup (succeeded)
5

Oh, of course. Using the local database. You’re correct. My mistake :slight_smile: