Many of these different matters (e.g. transfer rate) affect decisions on performance tuning, although there is no formula for it and it’s all educated guesswork based on what is known.
I disagree. Smaller files exist on many systems. I often have smaller .txt
, .pdf
, etc. Checking system altogether, I found over 1 million below 1 MB, but most aren’t things that I don’t backup.
If you’re saying the default is too small for some of today’s expanding backups, I agree with that, however I’m not a maintainer. Changing its default also makes it a bit unclear what’s really used.
What else is not clear? I made a test backup with a 5 byte file recently. Its block was 5 bytes not 100 KB, because its data ran out. A smaller block is typically left at the end of even very big files because these are fixed size blocks unless they can’t be, such as at the end of a file (generally).
Cryptographic hash function is what hash means. It’s sort of a 256 bit unique identifier of a block, meaning you can’t, for example, pack 10 blocks into a bigger one, as its hash can’t be calculated easily. Originally, the hash of a block is calculated at the time when a block is read for its backup.
exists, but it might be a bumpy ride for a motivated and technical person, in order to get it going.
You want to over-simplify a complex problem.
I posted two suggestions, based on not much, as it has not been benchmarked on any system, especially not yours. I don’t know your file counts or your file sizes (sounds like most are large).
Unless you have a whole lot of files (seems less likely based on the comment on small files, but your log file will say exactly what you have, if you can get to it), your choice of making blocksize identical to Remote volume size will probably work. You have been warned you’ll see small files.
There might also be something else running strangely here, but fewer blocks should help things.
EDIT:
Second thought.
I used the Everything search to count non-empty, < 1 MB files. Any idea what you actually have? Drawback of files smaller than blocksize plus blocksize = Remote volume size is only 1 block fits, resulting in the small dblock issue (not hundreds of bytes, but whatever the file size actually was). Running at blocksize = Remote volume size is rare (I think). Default 50 MB / 100 KB is 500 times.
My previous proposal was “5 MB, maybe 10”, but you could maybe push up to 25 if you like that, just so you can get several smaller files into a dblock. If all the files are big, the plan may change.