Block Size for Cloud Backup

FWIW, I’ve carefully considered block size as I’m using B2 also, using one account to back up about 5 different machines at this point, most of them set to daily backups.

B2 only starts charging you if you exceed 1GB in daily download bandwidth. After doing an initial backup of 50 megabyte files per Duplicati’s default, I decided that’s much too small, as it creates tons of small files in the backup set, and I’m pretty sure it bogs things down as it has to take a break from uploading every 5 seconds to create a new backup volume. I toyed around with 500 megabyte volumes briefly but then realized that even 2 verifications would then consume the full day’s free d/l bandwidth allocation, and I’ve been doing lots of cycles as I start up as I’ve been adding chunks to my main backup set (a few GB at a time, and so on).

I eventually decided on 200MB volumes for my main PC with the biggest data size, and for other computers I’m backing up to the same account have done anything from 150 to slightly over 200 MB. I’ve only exceeded the free d/l allotment a few days, and we’re talking about an extra $.01 or so on those days, so nothing that’s gonna have any real impact (and also nothing that’ll be routine). The 200MB size seems to work well and has sufficient speed, though of course I’d be curious to see someone run benchmark tests across a range of sizes to get official numbers.

I also have a second backup set on my main PC which goes to a 3TB external drive, and since there’s no download allotment for local drives, I set the volume size there to 2GB, and that’s working pretty well too. Obviously for B2 there would be lots of cons to doing that (any sort of verify or restore operation would start incurring fees instantly), but no such concerns for local of course.

I’ve noticed B2 stores the SHA1 hashes of all individual files, and Duplicati supposedly has some sort of capability to use these for verification (instead of and/or in addition to physically downloading the backup volumes) - but from what I’ve read it doesn’t actually do this yet. I’m hoping this feature gets implemented sometime soon as it could get us a lot more verification of our remote backup sets for minimal download cost. I said in another thread that I’d like the option to have it download 1 set to verify and do hash checks on ~4 other sets everytime a backup runs, where both of those numbers are adjustable by the end user. Hopefully it won’t be that hard to do once the main functionality is implemented.

2 Likes