Duplicati will sometimes do a compact operation where it will download and repackage dblock files that now have wasted space inside them due to retention settings. This makes your backups take less space on the back end, but it should probably be disabled if you have a download cap:
What happens when the download cap comes into play? I have it set to not download anymore data. It seems that Duplicati completes just fine. Does that mean it successfully compacted some blocks before stopping?
Just wondering if that over time it will work its way through compacting. I didn’t hit the limit at all for the month of July, but did hit it for August, May and June.
Interesting - I would have expected it to cause an error if Backblaze stopped Duplicati from downloading a file.
You will probably be fine if you just turn off auto compaction. You could compare Backblaze storage cost to the bandwidth cost for downloading data to see if it’s worth it…
If that refers to “BytesDownloaded: 118184807”, that’s just 118,184,807 which seems about right for a 100 MB dblock, a dindex, and a dlist. Verification samples are in sets of these three. See The TEST command.
If you had a compact, it should have stats in this log file somewhere, including its own BytesDownloaded. Delete of a fileset (version) usually precedes a compact (otherwise there’s no point), and it did delete one. Compacting is an ongoing thing. You keep deleting old versions (retention policy), you keep freeing space. Duplicati won’t modify uploaded files (clouds often don’t allow it), so it creates new dblocks from leftovers.
BackBlaze is reporting large daily downloads when using Duplicati shows some ways to do smaller more frequent compacts if that suits you, but first thing to do is to check logs to see if you can find compact info showing it was the culprit, then you do your storage/bandwidth cost comparison if you’re after lowest cost.
I’m beginning to think there is something wrong with Backblaze notifications. I do a backup everyday starting at midnight. I got the 1GB warning today, for the backup on the 19th. When I look at Backblaze it says my total August download is 1GB.
So if this is correct I can say that the previous 18 backups didn’t download any data? I doubt that. . .
If you wanted an independent idea of how much data is downloading, some routers have statistics, and Windows might too. Of course you’d want to do the backup without any other programs reading data…
Does B2 really have a download “limit”? Or are you just talking about the free 1GB/day download?
I also use B2 and don’t remember getting any notifications at all about how much I download. Was looking at their pricing and see that they charge $0.01/GB for downloads (after the initial free 1GB).
I filed a support ticket with backblaze asking how my monthly download equals my daily when I know that I do backups everyday at midnight. And if they say last night hit 1GB how did all my other backups complete with 0 downloads.
Their answer was to quote and point to their FAQ (Which I provided in my question). Gotta love first level support.
As the Caps & Alerts are for daily use, will signal daily (or not, if the daily usage was not high enough).
Then, the averages of use over the month are calculated and that is the charge incurred.
I also looked at my latest run and it errored out because it failed to read. I removed my daily alert and it seems to be proceeding nicely. Duplicati is just downloading over a gigabyte of data during the download.