Complete, incremental or differential

sisytec · August 25, 2017, 6:46am

Hello.

I would like to know if I can choose between complete backup, incremental backup and differential backup, and how I have to do it.

Thanks and have a nice day

sisytec · August 25, 2017, 11:04am

I found it myself. Solved

tophee · August 25, 2017, 11:36am

It would be nice if you could provide your solution here so that others with the same question can benefit from it.

19wolf · August 30, 2017, 11:05pm

What did you do to solve this? I’m having the same problem!

drakar2007 · August 31, 2017, 2:41am

What problem are you actually having?

The impression I get is that Duplicati (2.0+) is using a compression technology that essentially incorporates incremental / differential backups in its core backup scheme, basically negating the need for separate choices between complete/incremental/etc. In other words, the backup fileset always represents a “complete backup”, but individual backup runs are in and of themselves incremental and/or differential as needed.

sisytec · August 31, 2017, 12:57pm

Hi @tophee, @19wolf and @drakar2007.

First of all, sorry for my delay (too much job).

Last week I read that Duplicati 2.0+ won’t do “incremental” or “differential” backups any more. Every backup is done as “complete”.

I’m not sure how, but my backups delay like incremental, but they are complete.

kees-z · August 31, 2017, 1:36pm

Duplicati goes a step further than most traditional backup programs.
In most traditional backup programs you can choose between making a full, differential or incremental backup. All backup methods have their own pro’s and con’s. Full backups are easy to restore from, but will cost a lot of storage. Incremental backups are relative fast (only new and changed data is written to the backup target), but restoring can be complex, because every incremental has to be played back after restoring from the most recent full backup.

Duplicati combines the advantages of all backup methods. It behaves as incremental when making a backup: only changed data is sent to the backup destination.
When doing a restore, all backups look like full backups have been performed every time: every restore point contains all files and folders of your backup source. When restoring a folder from your most recent backup, only one restore action has to be performed, in contrast to a “real” incremental backup.

In summary: Duplicati indeed won’t do incremental and differentials backups, it does this better and smarter.

wma · February 10, 2018, 11:18am

The 1 Million dollars question, how reliable is the whole thing? Let’s say that I set backups to run daily at 20:00 with a setting to remove all versions older than one month. Let’s say that I need to recover my files after some 9 months and 21 days and 5 hours … Will it work? Will it work to restore the files into a new machine (new Windows installation)? Thank you.

cpo · February 10, 2018, 11:47am

You always will be able to restore your files.

let’s play with your retention plan above: as long as your backup is running you will be able to restore all files that were on your machine (and backuped) a month old and younger. If you delete a certain file on your machine it wll be gone from your backups after a month (normal behaviour with backups and a 1 month retention plan). BUT if your backup month is, let’s say, january 2018 and your backup stops there - then you will always be able to restore the files from january - even in september 2018. (but your backup had to stop in january).
As long as your new machine has access to your old backup data you can restore it directly from the backup back to your new machine.

Hope that helps

wma · February 10, 2018, 3:33pm

Thanks @cpo … I will be saving the backups files on remote Linux server using ftp, my question is it possible to verify the integrity of the backups server side? I mean on the Linux shell? And possibly list the backed up files?

cpo · February 10, 2018, 3:45pm

I would think this is possible if you install duplicati at the remote server. You don’t configure any backup-jobs there - it’s just for integrity checks. You would have to use the cli over there to run a verify job on the server. You could also use the cli to run a list-files job, but i inagine this could be a lenghty task because the database for the job will still be on the local source running the backup. without this database the duplicati on the remote server would have to check everything direct from the backup files…

But i admit, further than this is a bit out of my knowlegde base.

kenkendk · February 10, 2018, 9:54pm

You can set the --upload-verification-file option to have it upload a JSON file which lists all the expected files and their hash values. There is a script here that can read this file and verify that it is all fine:

github.com

duplicati/duplicati/blob/master/Tools/Verification/DuplicatiVerify.py

#!/usr/bin/env python

"""
This file is a standalone python script that is tested with python 2.7 & 
python 3.8
If Duplicati is producing a backup with the option --upload-verification-file,
it will produce a *-verification.json file on the backend, which can be verified
by this script. Simply run this script with the path to the backup
folder as input, and it will verify all *-verification.json files in the folder.
"""

from __future__ import print_function
import sys
import os
import string
import json
import base64
import codecs
from hashlib import sha256
import argparse

This file has been truncated. show original

This is not possible (at least not on an encrypted copy) as that would allow the remote server to read your data. If the backup is not encrypted, you can read the dlist files and extract the file-listing from that.

Alternatively, you need to install Duplicati on the server.

wma · February 11, 2018, 7:30pm

@kenkendk, thank you very much for the valuable information.