Custom Backup Retention - Am I Understanding Correctly?

patg84 · December 12, 2022, 10:07am

I propose 3M:1D,1M:1W,3Y:1M

Now from my understanding this means the following?

Backup once a day and hold for 3 months (90 copies, one per day)
Backup once a week and hold for one month (4 copies, one per week)
Backup once a month and hold for 3 years (36 copies, 12 per year)

Does the backup retention policy flow on a first in first out basis? For example I start backing up 30 days worth, January 1st until Jan 30th. On the 31st day of January it would delete January 1st’s backup correct, then Jan 2nd, Jan 3rd, so on and so forth as time moved on? S̶o̶ ̶t̶h̶i̶s̶ ̶c̶o̶u̶l̶d̶ ̶b̶e̶ ̶c̶a̶l̶l̶e̶d̶ ̶a̶ ̶r̶o̶l̶l̶i̶n̶g̶ ̶s̶c̶h̶e̶d̶u̶l̶e̶?̶

Hopefully that makes sense.

ts678 · December 12, 2022, 1:29pm

This post seems to be suggesting that retention schedules backups, but schedule screen does that. Retention (any of the options) controls the retention, meaning what is retained rather than removed.

Smart and custom retention thin backups to increasing time interval between kept ones as they age.
This reduces storage use, however too long an interval may miss all copies of some short-lived files.

Duplicati.CommandLine.exe help retention-policy
  --retention-policy (String): Reduce number of versions by deleting old
    intermediate backups
    Use this option to reduce the number of versions that are kept with
    increasing version age by deleting most of the old backups. The expected
    format is a comma separated list of colon separated time frame and
    interval pairs. For example the value "7D:0s,3M:1D,10Y:2M" means "For 7
    day keep all backups, for 3 months keep one backup every day, for 10
    years one backup every 2nd month and delete every backup older than
    this.". This option also supports using the specifier "U" to indicate an
    unlimited time interval.

Above should also note the implied deletion after the longest time frame, which “U” can eliminate.
That was a late addition, and there’s lots of catching up to do in the help and manual (issue filed).
Volunteers to help with messages, manuals, code, and anything are scarce, and progress is slow.

patg84 · December 12, 2022, 6:52pm

I’ve updated the original post. So this, “7D:0s” means to keep all backups for the past 7 days including manual backups that are not part of the normal schedule?

ts678 · December 12, 2022, 7:05pm

Yes, all backups scheduled on Schedule screen or run manually. Finished backups are not categorized. Possibly you will find some oddities if you run a partial (using Stop button) and then restart that to finish.
Current syntax would probably write “7D:0s” as “7D:U” but it’s the same thing - no minimum interval set.

patg84 · December 14, 2022, 10:16am

Any idea why this happened?

Started a backup job on 12.12.22 with 3M:1D,1M:1W,3Y:1M. Job completed successfully. Job runs at 2:00am.
I changed 3M:1D,1M:1W,3Y:1M to 3M:0s,1M:1W,3Y:1M on 12.13.22 before the 2nd job ran on 12.13.22 around 1:00am.
Job runs again on 12.13.22 at 2:00am according to 3M:0s,1M:1W,3Y:1M. Job completed successfully.
Job runs again on 12.14.22. Completed successfully.
Noticed the backup for 12.13.22 is gone.
Duplicati reports two backups exist at this point. One from 12.12.22 and one from today, 12.14.22.

ts678 · December 14, 2022, 1:30pm

1M:1W is the shortest time frame (backup age), and you asked for minimum interval of 1 week.
Retention deletes happen after backup, but backup just taken is exempt from retention deletes.
Within the 1M time frame, backups are tested in order taken, so 12/12 sets start of 1W interval
which saw 12/13 showing up too soon, so 12/13 got deleted. All this is logged. An example log:

2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-StartCheck]: Start checking if backups can be removed
2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-FramesAndIntervals]: Time frames and intervals pairs: 1.00:00:00 / Keep all, 14.00:00:00 / 08:00:00, 28.00:00:00 / 7.00:00:00, 365.00:00:00 / 31.00:00:00
2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupList]: Backups to consider: 12/14/2022 7:08:47 AM, 12/13/2022 7:50:03 AM, 12/12/2022 9:50:05 PM, 12/12/2022 1:50:00 PM, 12/12/2022 5:10:48 AM, 12/11/2022 3:50:00 PM, 12/11/2022 7:11:05 AM, 12/10/2022 3:50:00 PM, 12/10/2022 7:11:33 AM, 12/9/2022 3:50:00 PM, 12/9/2022 7:06:32 AM, 12/8/2022 3:50:00 PM, 12/8/2022 7:09:57 AM, 12/7/2022 2:50:04 PM, 12/7/2022 6:50:04 AM, 12/6/2022 3:50:03 PM, 12/6/2022 7:06:18 AM, 12/5/2022 3:50:00 PM, 12/5/2022 7:46:47 AM, 12/4/2022 5:52:26 PM, 12/3/2022 7:02:52 AM, 12/2/2022 3:50:00 PM, 12/2/2022 7:13:05 AM, 12/1/2022 3:50:14 PM, 12/1/2022 7:50:00 AM, 11/30/2022 3:50:03 PM, 11/30/2022 7:34:07 AM, 11/29/2022 7:50:04 PM, 11/29/2022 5:50:00 PM, 11/29/2022 4:50:01 PM, 11/29/2022 1:50:04 PM, 11/25/2022 2:50:01 PM, 11/22/2022 3:50:00 PM, 11/8/2022 6:35:03 AM, 10/3/2022 6:28:06 AM, 8/27/2022 6:05:27 AM, 7/21/2022 7:50:03 AM, 7/20/2022 4:50:07 PM, 7/20/2022 6:50:06 AM, 6/13/2022 3:50:00 PM, 5/31/2022 6:50:03 PM, 5/23/2022 2:09:32 PM, 5/8/2022 11:50:00 PM, 4/24/2022 10:03:48 AM, 4/2/2022 9:32:12 PM
...
2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsToDelete]: Backups outside of all time frames and thus getting deleted: 
2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-AllBackupsToDelete]: All backups to delete: 11/30/2022 7:34:07 AM
2022-12-14 07:51:34 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteRemoteFileset]: Deleting 1 remote fileset(s) ...

Typically one writes these with increasing time frames because it’s easier for humans to read.
Above says up to 1 day aged there is no minimum interval, then up to 2 weeks thin to 8 hours,
then up to 4 weeks thin to 1 per week, then up to 12 months thin to (not imaged) 1 per month.
Anything older than the longest time frame (12 months, written far right typically) gets deleted.

log-file=<path> log-file-log-level=information to get the above (copied out of profiling level log).

EDIT:

Here are the profiling lines that I deleted from between the information lines, if that helps more:

2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-NextTimeAndFrame]: Next time frame and interval pair: 1.00:00:00 / Keep all
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsInFrame]: Backups in this time frame: 12/14/2022 7:08:47 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/14/2022 7:08:47 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-NextTimeAndFrame]: Next time frame and interval pair: 14.00:00:00 / 08:00:00
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsInFrame]: Backups in this time frame: 11/30/2022 3:50:03 PM, 12/1/2022 7:50:00 AM, 12/1/2022 3:50:14 PM, 12/2/2022 7:13:05 AM, 12/2/2022 3:50:00 PM, 12/3/2022 7:02:52 AM, 12/4/2022 5:52:26 PM, 12/5/2022 7:46:47 AM, 12/5/2022 3:50:00 PM, 12/6/2022 7:06:18 AM, 12/6/2022 3:50:03 PM, 12/7/2022 6:50:04 AM, 12/7/2022 2:50:04 PM, 12/8/2022 7:09:57 AM, 12/8/2022 3:50:00 PM, 12/9/2022 7:06:32 AM, 12/9/2022 3:50:00 PM, 12/10/2022 7:11:33 AM, 12/10/2022 3:50:00 PM, 12/11/2022 7:11:05 AM, 12/11/2022 3:50:00 PM, 12/12/2022 5:10:48 AM, 12/12/2022 1:50:00 PM, 12/12/2022 9:50:05 PM, 12/13/2022 7:50:03 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 11/30/2022 3:50:03 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/1/2022 7:50:00 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/1/2022 3:50:14 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/2/2022 7:13:05 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/2/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/3/2022 7:02:52 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/4/2022 5:52:26 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/5/2022 7:46:47 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/5/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/6/2022 7:06:18 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/6/2022 3:50:03 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/7/2022 6:50:04 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/7/2022 2:50:04 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/8/2022 7:09:57 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/8/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/9/2022 7:06:32 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/9/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/10/2022 7:11:33 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/10/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/11/2022 7:11:05 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/11/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/12/2022 5:10:48 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/12/2022 1:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/12/2022 9:50:05 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 12/13/2022 7:50:03 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-NextTimeAndFrame]: Next time frame and interval pair: 28.00:00:00 / 7.00:00:00
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsInFrame]: Backups in this time frame: 11/22/2022 3:50:00 PM, 11/25/2022 2:50:01 PM, 11/29/2022 1:50:04 PM, 11/29/2022 4:50:01 PM, 11/29/2022 5:50:00 PM, 11/29/2022 7:50:04 PM, 11/30/2022 7:34:07 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 11/22/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 11/25/2022 2:50:01 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 11/29/2022 1:50:04 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 11/29/2022 4:50:01 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 11/29/2022 5:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 11/29/2022 7:50:04 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-DeletingBackups]: Deleting backup: 11/30/2022 7:34:07 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-NextTimeAndFrame]: Next time frame and interval pair: 365.00:00:00 / 31.00:00:00
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsInFrame]: Backups in this time frame: 4/2/2022 9:32:12 PM, 4/24/2022 10:03:48 AM, 5/8/2022 11:50:00 PM, 5/23/2022 2:09:32 PM, 5/31/2022 6:50:03 PM, 6/13/2022 3:50:00 PM, 7/20/2022 6:50:06 AM, 7/20/2022 4:50:07 PM, 7/21/2022 7:50:03 AM, 8/27/2022 6:05:27 AM, 10/3/2022 6:28:06 AM, 11/8/2022 6:35:03 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 4/2/2022 9:32:12 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 4/24/2022 10:03:48 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 5/8/2022 11:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 5/23/2022 2:09:32 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 5/31/2022 6:50:03 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 6/13/2022 3:50:00 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 7/20/2022 6:50:06 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 7/20/2022 4:50:07 PM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping partial backup: 7/21/2022 7:50:03 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 8/27/2022 6:05:27 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 10/3/2022 6:28:06 AM
2022-12-14 07:51:34 -05 - [Profiling-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-KeepBackups]: Keeping  backup: 11/8/2022 6:35:03 AM

A profiling log is huge. If you need extreme detail, maybe use log-file-log-filter to see RetentionPolicy.

patg84 · December 15, 2022, 5:08am

Thoroughly confused. There’s nothing in the manual from what I can find explaining the Custom Retention Schedule or showing in depth examples.

I’m not sure how to quote two things in the forum reply but why is it looking at the shortest time frame (backup age) first? That part comes second in the policy. I’m under the impression that the policy is read from left to right.

Keeping things simple, assuming that the policy was 3M:1D,1M:1W,3Y:1M.

If I told it to keep 3M:1D, that means keep 1 copy per day, every day, for the past 3 months. I should have roughly 90 backups. Confused as to why it deleted the 13th. I should have had the 12th, 13th, and 14th at that point. Wouldn’t it have waited a week before attempting to delete a backup according to the next part in the custom retention policy, 1M:1W ?

I’m wondering what it’ll do tonight (actually tomorrow morning at 1:00am).

Can you show me your backup retention policy then show me a screenshot of the restore page? Maybe I’m confused as to what will show up in the restore list.

ts678 · December 15, 2022, 12:58pm

GUI smart and custom backup retention aren’t covered #83 is the open issue on this, linked earlier.

Wrong impression as far as I know, but you can cite reliable sources if you like. Here’s source code:

github.com

duplicati/duplicati/blob/beaf03562fdcf4425e962085bdf7175d6a465f49/Duplicati/Library/Main/Operation/DeleteHandler.cs#L319-L323


      
          // Collect all potential backups in each time frame and thin out according to the specified interval,

          // starting with the oldest backup in that time frame.

          // The order in which the time frames values are checked has to be from the smallest to the largest.

          DateTime now = DateTime.Now;

          foreach (Options.RetentionPolicyValue singleRetentionPolicyOptionValue in retentionPolicyOptionValues.OrderBy(x => x.Timeframe))

This was already provided in the post above, along with how to get logs, my short log, my long log.
Thanks to my saved databases (a special setup…), here is how Restore list looked before that run:

Here is how it looked after that run:

Job log shows the deletion, also seen in previous post and visible in Restore list in current post.

This was explained thoroughly earlier. Please review everything posted, with correct understanding of

EDIT:

to clarify, that will work for human readers when policy is written as it typically is written per this:

and if you write it that way, it matches Duplicati too.

EDIT 2:

I just noticed that you proposed a different policy retroactively, then tied it to a previous backup.

and that’s the one I explained. What’s deleted is gone. That’s history and isn’t going to change.
You don’t say what you actually have now, but 3M:1D,1M:1W,3Y:1M would thin out backups to
1 week apart for first month, with new backup always kept, so 12/13 would disappear that way.

EDIT 3:

This text shows another potential misunderstanding. By saying :1W you set a thinning interval.
Typically more recent backups have more value in near versions, so the interval is set smaller.
By the thinking of your text, would smaller interval mean shorter wait? This is also not a wait…

EDIT 4:

Backup frequency and duration in Back up your Mac with Time Machine might clarify this.
The idea is the same (I think), but of course Duplicati uses different screens and word usages.
Duplicati is also very configurable. You see my use of a lowercase h for hours. s is seconds…
If you understand the concept of frequency, you have one for backups, and more for retention.

Lerrissirrel · December 15, 2022, 6:44pm

In case ts678’s responses don’t clarify it for you, here’s another way to describe what I think is your misunderstanding of what it’s actually doing.

Admittedly, I did not test any of this out, and if I’ve said something that contradicts what ts678 said, listen to ts678.

Did you interpret 3M:1D,1M:1W,3Y:1M policy as:

For the last three months, keep one backup per day
For the whole month four months ago, keep one backup per week
For the 3 years going back before 4 months ago, keep one backup per month

Or using an example as if today were November 30th (for simplicity), did you expect:

For September, October and November, keep one backup per day
For August keep one backup per week
For August three years ago until July of this year, keep one backup per month
Anything older than August 3 years is deleted

For a total of (roughly) 90 days * 1 + ( 1 months * 30 days ) / 7 + 36 months * 1 = 130 total backups retained at any one time?

Note that Duplicati may treat “1 month” as “30 days”; not sure about that.

I don’t think the timeframes are additive like that. Meaning the 1M is not the month AFTER the 3M. The 1M is this past month, which the 3M includes. All the timeframes (values before the colon) start relative to “now” - the day that Duplicati is looking to thin the backups. So I think 3M:1D,1M:1W,3Y:1M is actually:

For the last 3 months, keep one backup per day
For the last month, keep one backup per week
For the last three years, keep one backup per month

The example of this (again, starting November 30th for simplicity), turns into:

For November, keep one backup per week
For September, October, November keep one backup per day (except that rule #1 already removed all but one backup from each week)
For November three years ago through August of this year, keep one backup per month
Anything older than 3 years ago is deleted

You’ll end up with: ( 4 months * 30 days ) / 7 + ( 36 - 4 ) months * 1 backups = 16 + 32 = 48 backups

I think the first rule would, essentially, be ignored because it’s superseded by the second rule. It would say “For the prior month I should keep one per week and delete all other backups from each week”, so at that point, saying “For the prior 3 months I should keep one per day” can’t actually be done since it was already thinned to one per week.

To get what you probably wanted out of 3M:1D,1M:1W,3Y:1M, I think you need: 3M:1D, 4M:1W, 3Y:1M. This means:

For the last 3 months, keep one backup per day
For the last 4 months, essentially ignoring the last 3 months because they’re covered by the first rule, keep one backup per week.
For the last 3 years, ignoring the last 4 months, keep one backup per month
Anything older than 3 years ago is deleted

Going to my trusty example, if today were November 30th this would mean:

For September, October and November, keep one backup per day
For August, keep one backup per week
Starting November 3 years ago, through July of this year, keep one backup per month
Anything older than 3 years ago is deleted

Switching to 3M:0s,1M:1W,3Y:1M gets you, essentially, the same as 3M:1D,1M:1W,3Y:1M because the first rule is still superseded by the second rule. You’ll still keep, at most, one backup per week.

Hopefully this is not more confusing (and I really hope my interpretation is correct!)

Regards
L

ts678 · December 15, 2022, 9:09pm

That’s right. Think of them as age of some backup:

Gray Gradient Color Palette is a visual example. If you view “now” as right edge going right, notice how shading becomes lighter as you move left (older in time). Think of thinner ink meaning thinner backups. Left end of the illustration has lightest gray, and to left of that is edge of the picture. Think “no” backups.