Hi, new user of Duplicati, trying to get away from Crashplan. I tried a test backup of some video files to Backblaze B2, but got errors of the form “400 - bad_request - file size too big”. I think this is coming from B2 because some of the files are larger than 5 GB. B2 has a relatively new API for large file support that I think would solve the problem. Does Duplicati 2 support B2’s large file API, and if so, how do I enable it? Thanks.
If this is a new API feature to B2 then most likely Duplicati doesn’t support it yet.
However, I’m confused as to how you’re getting 5GB files coming out of Duplicati unless one of these is in play:
- explicitly set a custom
--dblock-size=parameter to over 5GB (the default is 50MB) - have an EXTREMELY large file list (or long file paths) causing the “list of files” file to be overly large
Can you post the results of “Export… → As Command-line…” here (with private content such as user names, passwords, and hashes removed)?
D’oh! I just double checked --dblock-size and found it was set to 50GB instead of 50MB. I think I tried setting it to 1GB, then thought I returned it to its original value, but must have missed the MB/GB. I thought the issue was that some of the input files going into Duplicati were > 5GB, and somehow that was causing the failure. I’ll try it again now that the dumb mistake is corrected. Thanks!
2 Likes
FWIW, for B2 I’ve found that it might be worthwhile to up the dblock size (slightly) from the default 50MB. I’ve settled on a size ~200MB, which cuts down a bit on just how many dblock files get uploaded, while not being so big that the file that’s downloaded when the operation is complete uses up too much of the free 1GB daily download bandwidth.
Looking at your example, if B2 user were to use --dblock-size=200MB and --backup-test-samples=5 they’d chew through their 1GB daily bandwidth every - day assuming only 1 backup / day…
I wonder if it would make sense to have the UI show (or alert) the user about estimated daily (monthly?) TESTING bandwidth usage based on “--dblock-size * --backup-test-samples * daily/monthly backup schedule”?
Though it could be useful and people might be interested it shouldn’t be a priority in my opinion. For a backup the costs shouldn’t be a factor. Features like suggestions in frequency based on connection speed and change of local file frequency would be a better feature. 
1 Like
But the default is 1, which is what i’m using. I agree though, if we were testing more samples per backup job, it would chew through the 1GB (but exceeding the 1GB on any given day only costs pennies, so OK to do intermittently, it won’t start really hurting until it happens routinely). Also B2 has server-side SHA1 hashes on each chunk, so eventually my hope is Duplicati can do multipiece verifications using the hashes and only need to actually download 1 (or 0) chunks to do a “manual” check.
Maybe, but someone would need to sit down and do custom ones for each provider, right?
Yeah, I thought of that - and it’s not even that simple as different provider packages could provide different limits so it’s not really feasible to warn people they’re going to go over.
I was picturing more of a general information section somewhere in UI. Kind of an expanded source and destination size area that could include things like"
- expected bandwidth for tests
- estimated backup run speeds per-megabyte based on historical speeds (this could also be leveraged to provide est. completion times for currently running backups)
- estimated restore run speeds per-megabyte (again based on historical backup speeds)
It’s all “fluff” functionality wise, but can also help with the “what speeds should I expect” type questions.
Sorry about hi-jacking the original topic…