Retention Policy not applied correctly

Support
1

Hi everybody!

I have a little issue with custom backup retention policy
I have configured it as follow: 92D:1D,29W:1W,25M:1M
So I will have one backup per day for the past 92 days, one backup per week for the last six months and on one backup per month for a little bit more than two years.

The point is that duplicati do not keep a copy for each days:

image
image.png517Ă—740 39.3 KB

image
image.png319Ă—536 11.3 KB

On the screen shots you can see that duplicati did not kept a version for the 23, 17, 15, 11, 3 of January when a backup has been on these days. It seems that it kept a backup for each two days instead of a backup for each day.

Thanks for your help.

2

I could be mis-remembering here but I believe the retention rules are applied based on the END time of a backup.

This could result in what you are seeing with the following scenario:

  1. Backup starts evening the 23rd but ends morning the 24th.
  2. Backup starts evening of the 24th and ends still evening of the 24th.
  3. Retention rules see TWO backups on the 24th and remove the oldest (the one that started on the 23rd).
3

This seems likely. It’s definitely the biggest “issue” with the retention policy. It’s super hard to define a perfect fuzzy border making the backups fit within the policy.

It’s also possible the backup just wasn’t run that day, but it’s hard to tell without knowing the entire history.

4

I was assuming the “Result” items (shown below the restore list) were from the log show run times.

Most of my backups are in the early AM so pretty much always end on the same day as they start so I’ve not really noticed the described issue.

HOWEVER - there have been a few times I’ve done manual test backups in the evening only to see them disappear the next day. But since they were test backups I never cared enough to confirm my theory. :slight_smile:

5

Thank you for your quick answers.
I have check the logs, all not retained backups have ended the day they have start:

image
image.png692Ă—204 16.4 KB

image

image
image.png795Ă—234 15.1 KB

image
image.png751Ă—161 12.2 KB

image
image.png814Ă—149 12 KB

BTW, the warning is just because I was still using the obsolete “Verbose” option for log-level.

6

I was assuming the “Result” items (shown below the restore list) were from the log show run times.
It is correct:

image
image1600Ă—900 62.6 KB

7

Every backup listed as deleted incorrectly was less than 1 day from the previous one, thus was deleted according to the 1 day retention. 1 day is 24 hours or 1440 minutes or 86400 seconds. You can give the interval in those units with h m s respectively. Everything is converted to seconds. It’s not calendar time.

Custom backup retention unexpectedly deleted a backup says more. Use logging to see the processing.

I think it’s backup-to-backup time distance, with backups known by the usual time on Restore dropdown. Deletions of any sort happen after the backup, but the backup just made is exempt from some deletions.

2 Likes
8

Makes sense. Would rounding to nearest calendar unit be more “logical”?

9

Brilliant ts678!
All missing backup have been run less than 24h after the previous backup.

Basically, if I run only one backup per day (no more), I can sort this by using the following retention policy:
92D:1H,29W:1W,25M:1M

That being said, for people making more than one backup per day, it might be considered Pectojin comment

10

You can also use 92D:U if your intention is just to keep any backups for 92 days

11

So is “U” the same thing as “0s”?

12

It reads as “unlimited”, but yes 92D:0s would work too.

U:1D reverses the equation, though, keeping one backup a day for unlimited time. So equating U with 0s can be confusing.

13

Is H accepted as a placeholder for hours?
Something like: 92D:2H,29W:1W,25M:1M

14

Actually, now that I think about it, it’s lower case.
e.g.

30m:U,1h:600s,3h:15m,1D:1h,1W:1D,1M:1W,1Y:1M,U:1Y

Should showcase the options, although it’s obviously overly complicated to show many options.

Within the last 30 mins keep every backup.
Within the last hour keep one backup for every 600 seconds.
Within the last 3 hours keep one backup for every 15 minutes.
Within the last day keep one backup for every hour.
Within the last week keep one backup for every day.
Within the last month keep one backup for every week.
Within the last year keep one backup for every month.
For every year keep one backup.

As pointed out previously, it’s not very fuzzy, so it is time between the backups.
E.g. 3 backups in 3 hours may become 2 backups if the first two occur 30 minutes apart and the policy keeps one for every 60 minutes.

1 Like
15

Thanks a lot for your time, quick replies and explanations Pectojin!

1 Like
16

I knew that but had forgotten, thanks for reminding me! :smiley:

@Backarchy, are you OK with @ts678’s post being flagged as the solution?