Change Backup Destination during initial run

Support
1

Hi,

I have a decently sized backup set (500GB+) currently backing up to B2 and a slow upload speed (1mbps), I have come across a few other forum posts which suggest I can backup to a USB drive and then use another faster internet connection to seed the backup files, then once this is done change the destination back to B2 and Duplicati should pick up from there.

This is all well and good if I haven’t started the backup yet, but in my case I have already uploaded 100GB or so but the initial backup run hasn’t finished yet.

I would like to switch to performing a seed, but I don’t want to lose the files which have already been uploaded to B2, to save time. (I know that from a faster connection, it probably doesn’t matter if I do lose the files)

  • What would happen if I change the backup destination now to a USB HDD, before the initial backup has finished? Would it start again from 0%, or would it continue where it left off?
  • If it did start again from 0%, would the backup files be compatible with the original files uploaded during the job to B2?
2

Neither. Duplicati would complain about missing destination files.

There MIGHT be an advanced parameter you could use to tell it to ignore the missing files (maybe something like --no-backend-verification) but I don’t know how well things would play together.

If you do want to go this route I’d suggest making a test backup to a local drive, interrupt it (allowing it to finish the current upload), change the destination to a different local drive and let the backup finish (if it will). Then merge the two folders and see what happens.

My guess is you’ll have an issue with a single same-named file in both locations. Most likely if you delete it from both locations it will be recreated - but I’m not positive on that.

1 Like
3

Thanks for the response, just to provide an update I created a backup set to test this and long story short it seems to work properly the way I hoped it would work. I’ve included some information on my testing process below, and I may have also discovered a bug in the restore functionality (detailed below also).

To begin testing, after creating a new backup job I stopped the backup allowing it to finish the current upload (confirmed also that when it stopped, it had not uploaded all files yet), I then changed the destination to another local location.

When I first tried to start the job again after the destination change, I got the missing files error as you mentioned:
Found 19 files that are missing from the remote storage, please run repair

Attempting to run a repair copies across the dlist.zip.aes and dindex.zip.aes files which existed in the original destination to the blank destination (which would have caused a filename merge conflict later if I tried to merge both destination’s files), however the job still does not run unless I add --no-backend-verification.

From here, I decided to start my test fresh without running the repair, as having run the repair may cause more issues down the track when I merge the files. I deleted the backup job and backup files, and started fresh.

This time, without running the repair and with --no-backend-verification the backup started to resume in the new destination where it left off in the original destination.

The job uploaded the new dblock, dindex and dlist files (the filenames in this instance did not conflict at all with the previous destination).

Just to add, after the backup finished successfully in the new destination, and before merging the two destinations (and before changing the backup destination back to the original destination) I tried to run some test restores to prove that my files weren’t being uploaded twice. Originally I had an issue where a file that was backed up in the initial run to my first destination could still be restored even after I changed the destination to my second location. (The blocks for this file were only in the original location!)
I must have discovered a bug, as the original file path (source of the backup) still existed so I caught Duplicati copying the original source file to the restore location, instead of reading from the backup files. This was proven by renaming my source directory, which then made the restore error out due to cannot find the block file - and this was my expected outcome as it proves to me that the files processed when I started the job for the very first time were not backed up again after I changed destinations.

I could merge the two destinations without any conflicts. After changing the destination back to the original location everything seemed to run fine, restores worked from all restore points both on the original destination and the second destination. The restored files also did not have any issues, one of them was an installer which performs its own integrity check and that passed with no issues.

1 Like
4

Nice job on the testing and documenting - you sound flag your own post as the solution!

The “copying the original source file” restore thing is a feature, not a bug. No, really! It’s designed so during a restore if the local file exists and matches what Duplicati is being asked to restore the local file is used since that will almost surely be faster than pulling from a backup.

In an actual restore scenario most people would probably want the fastest process possible. But, understandably, it’s much less desirable for test restores. For those, you want to use --no-local-blocks:

5

Excellent thank you for clarifying! I can see how it’s a feature when we’re using the software normally.
Also handy to know that option exists for testing in the future.