Retention config setup (stacking arguments with same timeframe?)


#1

Hi there :slight_smile:

i just updated to the latest experimenal (2.0.3.1_experimental_2018-03-16) just to finally use my own retention plan :slight_smile: Thanks for everyone making this possible.

My question though is:

  • my jobs run 3 times a day (every 8 hours)
  • i want to keep these 3 backups per day for a week
  • then 1 backup per day for the next 3 weeks
  • then 1 backup every 2 days for the next 4 weeks
  • then 1 backup every week for the next 4 months
  • and then 1 backup every month for the next 6 months

That would add up to 12 months to keep backups.

Is it possible to achive such a retention plan? If so, how?
I do not know if i can combine more than one argument with the same timeframe (i.e 3W:1D, 4W:xxD).
What would (should) happen is i use something like the above?

Thanks for your help.


#2

You can add as many timeframes as you like. Retention policy is applied starting with the most recent timeframe until the oldest poijnt in time is reached. All older backup versions will be deleted (unless keep older backups is specified in the retention settings).
So your example would do this: “for the backup versions between today an 3 weeks ago, keep 1 backup of each day. For backups between 3 weeks ago and 4 weeks ago, keep 1 backup of every xx days”.

So to set the policy that you want, you should enter:
1W:8h,4W:1D,8W:2D,5M:1W:1Y:1M.
If you want to keep backups older than one year, add ,U:U. This will keep all backups (unlimited) for an unlimited time.


#3

Ah, thank you.
I wasn’t aware that there is an expression for hours (h). It’s not mentioned in the GUI.

What would happen if i say 1W:3D? Would that be equivalent to 1W:8h?
In the GUI explanation 7D:1D seems to be read as “keep 1 per day for 1 week”.
So 1W:3D would read as “keep 3 per day for one week” or in other words 3 per day?

Edit: No it won’t. :slight_smile: Just found out the “hard” way :wink:
So the second argument is not a “per”. It’s more a “time between kept backups”.


#4

Yes, it’s essentially for X [time] keep one backup every X [time]

With support for

s - second
m - minute
h - hour
D - day
W - week
M - month
Y - year

One “different” option isn’t really a time, it’s more of a concept

U:1D - for unlimited time keep one backup for every 1 day
1D:U - for 1 day keep unlimited backups

It’s also worth noting the time-frames are somewhat fuzzy, but they do not accumulate. So even though you keep one every 8 hours you don’t have to backup exactly at multiples of 8 (e.g. 8, 16, 24) to get 3 backups a day. But since they do not accumulate it means you could not have 3 backups within 1 hour even though they are the only 3 backups that day.

Might make sense to write a How To on this subject


New retention policy deletes old backups in a smart way
Request for smart retention of quarterly and annual backups
#5

Thanks, now things look a bit clearer to me :slight_smile:

But i can say that the retention works - within my first backup it thinned out my 156 versions down to 12 :wink:


#6

I would just write this as:
1W:U,4W:1D,8W:2D,6M:1W,1Y:1M,U:6M

changes:

  • 1W:U” - keep all backups for the first week, without worry about “3 per day” necessarily or maybe you kicked off an extra one at some point to capture some recent change or download; just all are kept. With retention policy enabled it won’t end up making much difference, storage-wise, if you keep one extra version for a week.
  • 6M:1W” - it sounds like this is closer to the original desired timespan, but otherwise just a superficial change
  • U:6M” - here i’ve added an example of what you could add if you want an unlimited final retention version thinned one further level to one backup per 6 months, i.e. 2 per year. If you don’t want them thinned down this far but you don’t want them to go away after 1 year, then you could change “1Y:1M” to “U:1M”, which would keep 12 versions per year indefinitely.

#7

That’s an excellent idea! :slight_smile:

I think i do have a much clearer picture. Thanks a lot y’all :slight_smile:
I’ll go with 1W:U,4W:1D,8W:2D,6M:1W,1Y:1M.

Unlimited makes no sense in my case. What is deleted by me or accident for longer than a year deserves to be gone :wink: And for versioning, i do need some older versions sometimes right after the change has happened - so a week U and 4 weeks 1D is more than sufficient.

Thanks again! :slight_smile:


#9

Yes, a MAN page with many examples is in my opinion necessary. I do not understand the syntax explanation.


#10

As confusing as it might appear at first you may find that on close inspection it’s just as confusing. :slight_smile:

For now the “man” page is pretty sparse Advanced Options - Duplicati 2 User’s Manual.