For me the 40 backups set via the GUI policy will be just fine, and I don’t mind that this will be the limit and more will be deleted, regardeless of any custom policy set via the advanced command line options.
I was just wondering whether the command-line switch will keep working? Could it be possible to set both an “custom policy” via the advanced command line options and via the GUI? Because that should not be possible?
I tried it but then the GUI seems unable to determine the correct policy and I need to remove all policies and clear cache to fix it.
Nice to have the --retention-policy option integrated in the GUI, but I guess at least the --keep-time option should stay available somehow when choosing “Smart backup retention” or “Custom backup retention”.
The retention policy for Smart retention is “1W:1D,4W:1W,12M:1M”. This leaves all backups older than 12 months untouched.
Combined with for example --keep-time=3Y, all versions older than 3 years will be deleted. As a result, backup versions will not continue to increase indefinitely.
In the current version, you have to choose between smart/custom backup retention and keep time/versions, preventing deletion of old backup versions from the GUI.
Ahhh - so “1W:1D,4W:1W,12M:1M” combined with --keep-time=3Y means that as the 12 months worth of monthly backups age into 13+ months they’ll just sit around as monthly backups until they become 3 years old at which point they’ll be deleted.
When will we see this in an “official” release? I am using 2.0.2.1, I’d prefer not to use a Canary stream, which I understand is a developpers mainline.
2.0.2.1 is still in Beta Phase, so I expect the canary stream to be alpha or even lower?
If you use the dropdown to apply the settings, it should clear the other (i.e. use either --keep-time or --retention-policy). If you use the Advanced options, it just applies those after the dropdown options.
No, it does not clear anything… I use the basic dropdown, set custom retention policy, and after save and re-visiting it again presents me keeptime=12M. I then used the advanced options, only added custom retention. After save and revisit, Duplicati show the custom retention in (5) General options and advanced options.
But when I look into “Export configuration” I can see it has stored both, Keep-time and retention-policy.
FYI, i’ve found the canary builds to be very stable. If you were hesitating in installing a Canary version, I’d say it’s unnecessary - even if you just want to install the current one to test the current features and then wait until major releases later.
It’s actually my experience that canary has the least problematic bugs, so being on canary might be safer than beta right now
Of course the problem with canary is that it’s not been tested by more than a few people before it’s released, so that assumption could change any time.
Right now I’m not sure how much the semantic difference matters. There has only been 1 experimental release since the beta, which was in august. I think the beta has some serious issues that haven’t been addressed despite fixes being upstream and in the canary.
I know it takes a lot of effort to maintain more frequent releases, but it’s always sad to have to explain to users that their issue on Github has already been solved but because their software is 5 months old they haven’t received the fix yet.
I’d love to take a conversation on release cycle. I think more frequent releases will allow Duplicati to reach a stable release much quicker. Many fixes are so simple they could be pushed directly into a experimental/beta release after a little testing/validation from someone other than the committer. Keeping new features in Canary for a while is good, but fixes solve problems that may be urgent to users and should be released as soon as possible.
I’ll happily donate time to help figuring out what goes into the releases or even build packages if that’s what it takes. I just don’t know how to help here.
I will just go ahead an admit that I am bad at pushing the releases and keeping a roadmap on track.
What usually happens is that we agree on some features/bugfixes that should make it into the next release. Then new bugs emerge and new fixes, and then I delay the release to get those into the next version, and we end up not having a new release.
Since the releases are signed with a certificate that is my personal one, I need to do the actual builds, and I will happily do so.
But we need someone who decides which fixes go into what version, and which issues are blocking a release. Alternatively we can do it with a fixed scheme, say a beta each 3 months, but I think that is more likely to break things.
I can give you commit access to the github repo. From there I think it is a matter of creating a branch (say releases/experimental_v2.xxxx), base it on some relatively safe point, and maybe cherry pick commits into that.
Once you are happy with the version, maybe we should start a topic in Releases and people can chime in with opinions (in case there is something that really should not go out).
Then I will be the “build-bot” and press the button once we have a “go”.
I think the milestone approach is the right way to plan and discuss releases, but I have been tied up (too much) on the concurrency branch, because it will also greatly improve code readability and make it simpler to introduce a number of larger changes that I have planned.
It looks like I have managed to get it working, but Travis is having some issues, so I cannot verify that it actually works.
Heads up!:
We decided to change the --retention-policy option a bit: It now deletes backups older than any timeframe by itself, so there is no need anymore to also use --keep-time. And to not have any confusing situations with people combining --retention-policy with --keep-time and --keep-versions, these are now all mutual exclusive, meaning you get an error if you have them combined in any way.
For those that might not follow the difference, here’s a quick summary of how --retention-policy cleanup handles “end of life” of a file / version:
any file still existing in the source will always have at least the most recent version in the backup even if that most recent backup is older than retention policy limits. (Unless --allow-full-removal is enabled, in which case backup versions older than retention rules allow WILL be removed from the backup even if the file still exists in the source.)
any file deleted from the source will subsequently be deleted from the backup when it reaches the retention policy limit. This means that if you want to keep files “forever” (like CrashPlan allows) you should make sure your retention policy includes a very long time frame. For example, adding “99Y:99Y” means a deleted (from the source) file won’t be deleted until the version in the backup is over 99 years old.
The “99Y” before the colon means keep “things around for 99 years”. The “99Y” after the colon means “keep 1 version for every 99 years”.
Wait, so this would mean that with a “1Y:1W” retention plan and --allow-full-removal enabled I would lose the ability to restore files that have not changed in the past year even they still exist on the system? I’m curious about when that use case would ever be desired.
As I understand it, yep. I personally wouldn’t do that, but if somebody had very limited destination space or wanted to simulate a “full backup” but for some reason didn’t want to use --keep-versions=1 I suppose it would do the trick.
I supposed it could be geared more towards manual tasks (single file cleanup and the like) than every day backup use. Though even then purge would probably be a better choice.
