Optimizing Online Storage

I use Backblaze B2 for online storage of Duplicati backups. B2 has an option to delete old versions of files stored in B2 buckets.

image

Since I’ve been using B2 for a while my storage costs are starting to rise. Not enough to need to sell the house or anything like that, but enough that the household CFO is starting to question the expense. Hence, I’m trying to see if I can reduce the costs somehow.

Unfortunately, I am woefully unaware of the way Duplicati creates & manages the .aes files it uses for backups. If I set the B2 retention to “Keep only the last version of the file” will that interfere with the way Duplicati stores its .aes files? Does Duplicati reuse the .aes files as it creates backups sets such that there might be multiple versions of an .aes file being stored in B2?

Thanks,

HG

I believe that Duplicati never modifies files on storage - it either creates new ones or delete unneeded.
The only reason why you would keep deleted files is to recover from accidentally deleting all of them.
I think the best option is to set days to keep them, maybe 2 weeks or so…

Personally I configure B2 to only keep latest version of each file. No extra versions or deleted item retention. Duplicati manages backup versions on its own and doesn’t use lifecycle options on the back end storage.

I use B2 as well there is a bigger cost in downloading files so if your are doing full verification’s or compacting files then it will increase the cost. On B2 I use no-compact option. If you have a bigger then standard dbblock size or if you are running multiple backup tasks every day it downloads one file by default to verify on each backup this could be add to your costs,

To answer the question I keep only the last version.

Some of below is going to be specific to Backblaze B2 pricing and screens, some is generic.

My Account → Caps & Alerts page will tell you where costs come from. It’s not only storage.
Downloads and possibly transactions can be billed when your use exceeds the free amount.

Reports page will give a longer-term view. Just looking at dailies may miss work of compact.

There’s some tradeoff. –no-auto-compact may cut download cost, but increase storage cost.

Running backups frequently won’t raise upload costs (which are free), but will do downloads.

B2 Buckets page gives size of all your buckets, if it does turn out that storage is the big cost.
Maybe you will find some things you no longer need at all now, but which you’re charged for.

Sometimes old backup versions get increasingly unimportant. Deleting some will save space.
Backup retention on screen 5 Options has a variety of options to keep versions under control.
Note the space will not immediately be lowered, as reclaiming the waste is done by compact.

Duplicati Home screen has the Source size and the Backup size for every backup configured.
Compression and deduplication can make first Backup smaller than Source files that it’s from.
Eventually version buildup will probably make it bigger. Detailed size information is in job logs.

Introducing Backblaze B2 Lifecycle Rules shows how to see if you actually store old versions.
This isn’t what Duplicati relies on, as many storage types don’t have it. Duplicati does its own.

My B2 Lifecycle Settings remain at the initial Keep all versions of the file (default).
Looking at names and icons on B2’s web UI, I’m not seeing any B2-level old versions present.

At Duplicati level, I’m using Custom backup retention similar to Smart backup retention.
New retention policy deletes old backups in a smart way was the announcement of the feature.