Removing files from backup

Support
1

I have backed up some files which I subsequently deleted. I would like to remove them from all the old backup sets to reclaim cloud storage space.

I did it manually with a purge but is there a way to automate it?

Also, with Smart Retention, will those files ever be purged? It sounds like they will live forever in the backup set.

2

See explanation of Smart retention policy:

Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.

You can also set custom retention. Example: 2D:U,1W:1D,4W:1W,36M:1M works fine for me.
You can find more explanation of retention policy in this post

1 Like
3

Thanks. So it sounds like I need to have time-keep set up, or is that part of Smart Retention?

4

I think, smart retention should work and keep the same amount of backups as custom retention set to 7D:1D,4W:1W,12M:1M

5

Thanks. I tried adding keep-time as your link suggested, but it produces an error stating that you can’t have both keep-time and a retention policy.

Looking at my older backup tasks none of them have backup sets older than 1 year, so I guess they are being removed and any files deleted longer than 1 year ago with them.

6

Yes, I agree with the @PeteM description and the quoted help text. The 2017 discussion is a bit dated.

Test smart retention policy #3008

We had some good discussions how the feature works, we have introduced that backups outside the specified time frame get deleted, and we had some more weeks with no complaints that anything was seriously broken. This should be sufficient to let this optional feature pass to “experimental”.

Feature/issue 3008 retention optimizations #3026

As discussed in #3008:

  • --retention-policy now deletes backups that are outside of any time frame. No need to specify --keep-time as well. Help text was changed accordingly.
  • --retention-policy is now mutual exclusive to the other retention options --keep-time and --keep-version. Help text was changed accordingly.
  • --retention-policy now complies with --allow-full-removal, deleting even the most recent backup, if it’s outside of any time frame

It’s automated if you can express it with one of the retention policies, e.g. are willing to say how long the backup should retain old versions of files to guard against unwanted deletions or changes. If it’s noticed soon that some files got backed up that you don’t want, an easier and safer-than-purge way is to delete entire unwanted backup version right then with The DELETE command. The latest version is always 0.

This method isn’t selective, so make sure that you leave enough recent backups to feel comfortable for other files, but from a storage point of view it might be more attractive than typing up space a long time.

Purge is fast too, but if you mis-aim a purge-these-files-in-all-versions you can seriously harm a backup.

7

Thanks. I wasn’t sure based on what the UI said:

Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.

The bit in bold is confusing, I took it to mean that there will always be at least one copy of a file that was backed up. I guess it means that there will always be at least one backup set, even if that set is outside the timeframe.

1 Like
8

Not quite. None of this is per-file. Running a backup makes a version that has the source files present then. There is a general safeguard against deleting all the versions. This can be overridden by allow-full-removal.

Unless overridden, neither the delete command nor retention will delete the last one, but you’d have to get pretty clever (or maybe unlucky) to get near that. Retention (smart or custom) keeps most recent backup, exempting it from the interval rules because it would make no sense to create it then immediately delete it.

That would take some hard work (or maybe an accident…) to get even close to. More likely the surviving version (without allow-full-removal) is the one that was just run , not the one outside all time frames.