Scheduling restore


#1

A common pitfall with backup is that it might look like it’s doing it’s job perfectly year after year when the truth is it might have partly stopped working a long time ago due to some minute change or error somewhere. Which you’ll either find out when you regularly test restoring or when you desperately need to restore.

I would very much like to be able to schedule restores. Since nobody does a manual test restore daily it would be very nice to have a regular basic restore scheduled that might catch some errors. Obviously you are never 100% sure if everything is backed up as it should neither with manual or automatic test restores. But it would be very very nice to enable this base test daily. Yes I know it has its limitations scheduling this but it’s a huge improvement over doing manual tests every six months or so.

I could have a small external script checking file sizes and then erasing the files for tomorrows scheduled run. Utimately I’d like to have the automatic restore to restore from different parts of available backups but that is a completely different beast. Just having scheduling added would be very very very nice.


#2

Not sure if this fits your need, but Duplicati already performs an integrity check after each backup task by downloading a number of backup files and analyzing the contents of these files. How many files are downloaded, can be set with advanced option --backup-test-samples.

The integrity check can be started manually with the test command.

Usage: test <storage-URL> <samples> [<options>]

  Verifies integrity of a backup. A random sample of dlist, dindex, dblock
  files is downloaded, decrypted and the content is checked against recorded
  size values and data hashes. <samples> specifies the number of samples to
  be tested. If "all" is specified, all files in the backup will be tested.
  This is a rolling check, i.e. when executed another time different samples
  are verified than in the first run. A sample consists of 1 dlist, 1 dindex,
  1 dblock.

Actually restoring a file for testing purposes should always be performed manually. If you rely on it, you will never be notified if the test restore doesn’t work at all or if the restored file is different from what you expected.


#3

That is good info! Thanks!


#4

You should be able to use your OS scheduler to fire off (directly or as part of a batch) a Duplicati.CommandLine.exe restore .... call.

In theory you could have a monthly batch file that restores, compares, notifies a different chunk of the backup each month.

Building the command line might take a little effort (I really wish there was an “Export to Commandline” option on the Commandline page!) but I don’t see why it couldn’t be done.

You can find a “full” export of the build in Duplicati.CommandLine.exe (for 2.0.2.1 beta) in this post:

But I’m pretty sure you’ll only care about this:


Duplicati.CommandLine.exe help restore

Usage: restore <storage-URL> ["<filename>"] [<options>]

  Restores <filename> to its original destination. If <filename> exists
  already, <filename> is changed to <filename-timestamp.extension>. To
  restore all files use "*" or leave empty.

  --overwrite=<boolean>
    Overwrites existing files.
  --restore-path=<string>
    Restores files to <restore-path> instead of their original destination.
    Top folders are removed if possible.
  --time=<time>
    Restore files that are older than the specified time.
  --version=<int>
    Restore files from a specific backup.

And in case you care, here are the Test options:

Duplicati.CommandLine.exe help test

Usage: test <storage-URL> <samples> [<options>]

  Verifies integrity of a backup. A random sample of dlist, dindex, dblock
  files is downloaded, decrypted and the content is checked against recorded
  size values and data hashes. <samples> specifies the number of samples to
  be tested. If "all" is specified, all files in the backup will be tested.
  This is a rolling check, i.e. when executed another time different samples
  are verified than in the first run. A sample consists of 1 dlist, 1 dindex,
  1 dblock.

  --time=<time>
    Checks samples from a specific time.
  --version=<int>
    Checks samples from specific versions. Delimiters are , -
  --full-remote-verification
    Checks the internal structure of each file instead of just verifying the
    file hash

#5

Yeah, I forgot all about running from the command line is possible. So it is doable already.

Time to make a small script :grinning::+1:


#6

Good luck!

If you get a chance let us know how it works out it works out for you in case others need a little encouragement. :slight_smile:


#7

Is Duplicati restore commands available on Ubuntu ?


#8

By default the .deb installer puts the Duplicati.Commandline.exe executeable as /usr/bin/duplicati-cli.

So you should be able to just use the restore commands with

duplicati-cli restore

#9

Great! Thanks for the info