So the first backup I did took a couple hours, but now each day the backup process only takes a few minutes.
Is Duplcati doing an incremental type backup, where is only backing up files that are new or have changed?
Is there other options like forcing a full backup everytime?
Yes, Duplicati is an ‘incremental forever’ backup system. Only new/changed files needs to be processed every time a backup is run.
When you go to restore data, Duplicati will show you a ‘full backup’ view for each backup snapshot. So even though it was technically an incremental backup, you’ll see all files listed under each backup snapshot version.
End result is you get the best of both worlds: quick backups (only changed data) and quick restores (only need to view one backup snapshot).
Technically, it is not an incremental backup, because the backups do not depend on each other.
In an incremental backup, there is a “base” and each incremental is the difference from some base (or another incremental). This forms a chain of backups, and can break in various ways.
Duplicati is more like a “differential” or “deduplicated” backup, in that it stores all unique data, and then on each backup figures out what it does not already know, and stores that.
There are no dependencies between versions (i.e. no dependency chains that can break), but they do share the stored bits so you save storage space.
Definitely true… in the classic sense it is neither incremental nor differential… not technically, anyway.
I still like to use the term ‘incremental’ because I think it’s what people are getting at when they ask the question - only changed bits are backed up between backups.
Some backup systems do what is called a ‘synthetic full’ at this point to remove the dependency chain and make the backup appear like a regular full. Duplicati does something like this, so in a way each backup is a ‘full’. That’s what I was getting at in my previous post… best of both worlds… quick backups, quick restores.
Well whatever you call it (Incremental Plus?), I like it!
I like it too! Have been using it over 2 years now and it’s my favorite backup system to protect data files.
I still use something else for complete system image level backups, since Duplicati is not intended for that purpose.
I dislike the term, because if you have used incremental backups, you realize that the incremental chains rely on a full backup, and are prone to breaking.
But other than that, you are right, it is similar enough to incremental in that it only takes changed items.
Duplicati backups are not differential, because differential backups also rely on previous backup data (the preceeding full backup). The only difference between differential and incremental backups is that the latter depends on the precceding full backup and the subsequent incremental backups.
Another characteristic of a classic differential backup is that changed data since the last full backup is uploaded each differential backup operation, until the next full backup.
These 2 characteristics do not apply to Duplicati: each backup is completely independent of any other backup and identical data is uploaded once.
From a backup-perspective, there is a similarity with incremental backups: only new and changed data since the last backup is uploaded. But that’s where the resemblence ends. A classic incremental backup contains new and changed data only. Every Duplicati backup operation results in a real full backup.
So I guess “deduplicated full backup” matches best what Duplicati does.
The backup process explained has a simple intro of a complicated backup defying conventional terms.
How the backup process works has technical details.
Block-based storage engine explains how the conventional full and incremental ideas no longer fit well.