Smart retention policy issues

Hi there !
Just a simple question, I configured my backup to use smart retention policy, so from what I’ve understood, it should keep:

  • 1 backup per day for the last 7 days (so 7 versions in the last 7 days)
  • 1 backup per week for 4 last week (so after these 7 versions, 4 versions)
  • 1 backup per month for 12 last month
  • 1 backup per year

Am I correct ?

Because when I look on my duplicati instance, I set the policy here : 1W:1D,4W:1W,12M:1M
But here is my history if I try to restore files:

  • 13/12
  • 12/12
  • 11/12
  • 10/12
  • 08/12
  • 07/12
  • 03/12
  • 24/11

On the last week, it should have keep one file per day, but the 09/12 is missing.
If I look into the logs, I can see the 09/12 the backup operation has been successfully achieved. The day after, the 10/12, I can see it deleted the version of the 09/12…I can’t understand why !
Can you help ?

The backup is programmed at 3am each day, and I double checked that the time is correct.

Thanks for your help !

This part isn’t in the GUI description (where is it from?), but you could add ,U:1Y if you want it.
Some may say 1 backup per year would be “smarter”, but I don’t think that’s the way it is now:

Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months.

How did you check time of backup? 1 day is an interval (not calendar-based) down to second.
Sometimes backup starts vary a bit, so 1W:1D,4W:1W,12M:1M for daily occasionally misses.
You could test a custom retention of 1W:23h,4W:1W,12M:1M to see if it avoids clipped dailies.

The Dec 9 backup is deleted if it is less than 86400 (24 * 60 * 60) seconds after Dec 8 backup.
Dec 9 when first made is exempt from deletion just after, but it loses the exemption on Dec 10.

If you kept a log-file log with at least information level, you would see Backups to consider.
Did all start at exactly 3:00:00 AM, or are any a bit late? Maybe some grace time could be used.

Indeed I missed the proper definition ! I’m okay with it. Thanks for the clarification.

Ok, I start to understand better ! When I say I checked the time, I connect to the container where Duplicati is hosted and I issue “date” command. So the timezone is correct for example. I also checked on the backup logs in Duplicati (it indicates which version is deleted).

I’m not sure which time I should take in order to do calculations but I guess it’s working as intended.
So OK, I’ll try with 23h instead of 1D, I guess it should work fine if the backup take less than 1hour, which should be the case.

I’ll keep you posted thanks for your help !

I think it’s based on start times, so backup length shouldn’t matter. Regardless, look at log.
For example, here’s an hourly-scheduled one sometimes being a little slow getting started:

2020-11-17 19:36:12 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupList]: Backups to consider: 11/17/2020 6:20:03 PM, 11/17/2020 5:20:00 PM, 11/17/2020 4:20:01 PM …

Being able to set schedule and retention in same units appears friendly until times drift a bit.

Indeed I think you’re right ! A few seconds can matter sometimes ^^

Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupList]: Backups to consider: 12/09/2020 03:00:02, 12/08/2020 03:00:03, 12/07/2020 03:00:00, 12/05/2020 03:00:02, 12/04/2020 03:00:02, 12/03/2020 03:00:02, 11/24/2020 23:58:14"