I’m using Duplicati to back up local files to my AWS account. I scheduled the backup to run automatically every day and set it to exclude backups older than 15 days.
When I try to restore a backup, I can see it working: the oldest backup is about 15 days old. But when I opened my AWS bucket, I can see zip files (…dblock.zip) with more than 15 days. Shouldn’t Duplicati automatically remove these files from the bucket?
Hi marcelo2605,
Welcome to the forum!
It takes some getting used to the concept that Duplicati uses vs what the usual “full” and “incremental” backup notion that most backup software uses. You can think of Duplicati as a single “full” backup the first backup you run, and then every one after that is an “incremental” - forever.
Duplicati doesn’t bother sending the same data again if it has already sent it. That includes data you haven’t changed, or if you’ve got new data to backup, but happens to be exactly the same as some other data you’ve already backed up. So let’s say you did a backup, didn’t change anything, then did another backup. Duplicati wouldn’t actually upload and of your data for the second backup. The Duplicati files holding your actual data, on the remote, would still show the timestamps from the first backup (except for some new ones, dlist I think, that hold the meta data about the second backup).
If you ever overwrite/delete every piece of data that existed at the time of your first backup, all the new data happens to have different hashes than any of the original backup, and it’s been more than 15 days, THEN you would find that Duplicati can, and will, delete those original dblock files since the data contained within them is not useful to recreate the logical image of the data for the point in time of any of your current backups.
So, you’re lucky that some old files are there! It means that you’re getting bandwidth/storage benefits by using Duplicati!
To build on the great post from @Lerrissirrel , see Features and The backup process explained which discuss how only changes are uploaded. So how does restore get old unchanged data? The old dblocks possibly are a part of that. Faster local sources such as original files and target files are also candidates.
Compacting files at the backend eventually cleans up old dblocks which hold too much now-unused data, meaning dblock files get compacted if source data churns. Dated dlist files should vanish on schedule.
How the backup process works and How the restore process works provide more technical explanations.
