The changes on my backed up source folders not only include changes on files and addition of files but also deletion of files.
If I would like to restore a backup I made with Duplicati (QNAP app version Duplicati 184.108.40.206_canary_2019-02-06) back to the source folders and this should reflect the exact state of the backup made including file deletions, then could I do this directly with Duplicati restore function or would I first have to erase my source folders completely and then restore the backup?
In other words, I would like to mirror my backup back to the source folders.
I don’t think Duplicati can do that directly, as it won’t delete files on restores. Such an option would seem to have some tension with Duplicati’s practice of restoring selected source files. One certainly wouldn’t want everything else on the drive deleted, but maybe everything else in a specific folder that’s specified in the UI.
Rather than delete source folders before restore, it would be safer to restore to a new area, then move the folders to where they belong, after deleting or renaming the curent folders that you’re seeking to mirror into.
This is basically the same thing as donig a complete restore in the original location after first deleting everything there. This is something that I would have like to avoid because my remote storage location (cloud storage) will provide a very slow restoring process.
In my case, I am asking about a single most current backup state that I would like to restore but one that I might want to restore via two separate backups, a fast one and a slow one (external HDD and cloud storage). It depends on the “disaster” scenario which one I would actually want to use (complete hardware crash of source location or loss of some files on source location).
Having zero copies of source files (i.e. deleting first) is less safe than having one while aiming to get two. Testing restores periodically can add confidence that it will work, but why push your luck? See below too.
Restore performance may also get dramatically worse after deleting source because Duplicati’s restore favors restoring file blocks from source files where possible, i.e. when the block to restore is unchanged. Specifying –no-local-blocks turns off this optimization, so is a good idea for a true test of remote restore.
The above file contains logging. Logging at verbose log level can show which way you restored, such as:
2018-12-16 21:00:02 -05 - [Verbose-Duplicati.Library.Main.Operation.RestoreHandler-FilePatchedWithLocal]: Target file is patched with some local data: C:\restore here too\short.txt
2018-12-16 21:00:05 -05 - [Verbose-Duplicati.Library.Main.Operation.RestoreHandler-PatchingMetadata]: Patching metadata with remote data: C:\restore here too\short.txt
For a dual fast/slow backup situation, the other topic mentioned xcopy /d as a possible way to update an older backup with newer files, but adding the further wish for “updating” with newer deletes limits options.
Programs exist for reconciling differences between trees of files. For example, WinMerge can show you newer deletions because the files will be in the old backup but not the new one. You can then delete files from the restored old backup. It can show file differences. You can propagate them from the new restore.
Ultimately the goal might be exactly what’s in the new backup, using the old one temporarily because it’s faster to restore. The simple solution might be to just use that as a read-only reference to be replaced by the new after the new is done. Old restore getting source files will help new restore get its source blocks.
You have a harder merge problem if you start actually changing files in the faster restore of older backup.
If you absolutely have to automate this instead of going into WinMerge, setting up the “View”, then acting, there’s a chance that rclone sync can do this, but it depends on whether this option overrules strict sync:
-u, --update Skip files that are newer on the destination.
These posts from rclone Issues give me hope that it can handle minor messiness, but I haven’t tested it.
Thanks for the detailed reply and pointing me to WinMerge because finding all excess files/folders manually after a restore from a backup without deletion of files/folders that were deleted in the meantime would be too difficult and time consuming.