I’d like to use DUPLICATI for production…
It seemed that DUPLICATI failed to restore in my testing conditions…
Am I Doing something wrong ?
I backed up 200Mb directory with about 1000 small sized files.
I split the backup file to 2 Mb, and set it backup into different hard drive…
in order to simulate disaster, I deleted the database and deleted one of the backup files.
in this case, DUPLICATI did not successfully rebuild a database (missing file error), the backup was useless.
Am I doing something wrong ?
How can I restore when one of the backup files is missing or corrupted ?
Duplicati tries very hard to guarantee that your backups work. When you delete remote files, you prevent the backups from working and producing the correct results (i.e. not all files can be restored).
The database can still be used for restoring, but you cannot run new backups on it. (If you want to run backups after volume loss, use the
purge-broken-files command to recover.)
If you have “less than perfect” remote data, you can add the advanced option
--no-backend-verification to prevent Duplicati from checking that your remote data is intact. This should be enough to get the restore running in the case you describe.
If that fails, there are two more options. One is using
--no-local-db which bypasses the local database (it builds a partial temporary database) and restores more “directly” from the files.
If that also fails, you can use the
Duplicati.CommandLine.RecoveryTool.exe to run a much simpler (and slower) restore process. Alternative to that there is a python script that does the same restore steps.
So it sounds like even if destination dblock (archive) files are missing the remaining data can still be restored? Awesome!
Keeping in mind this would be a terrible disaster scenario so getting ANY data back would be pretty impressive, if the standard 100k block size is used with the standard 50MB dblock (archive) size, does that mean that losing a single dblock (archive) file would mean anywhere from 1 to 512 (50MB / 100k) files would be un-restorable or that they’d be restored in a potentially corrupted state?
I would not use Duplicati myself if this was not possible
Normally, the volumes are either fine or totally broken. If they are totally broken, whatever data is in the volume cannot be applied. When restoring, the blocks will not be applied, leaving the files with “holes” (really just areas with zeroes).
You can use the “affected” command to query which files would be affected, should a given file be missing.
My plan to counter the broken volume scenario is to add parity files, such that a number of missing
dblock files can be recovered at the expense of uploading additional checksum files.