Please implement Blake2b (SHA-3 Finalist) file checksum hashing, besides or instead of sha256. It is blazing fast and even more secure than sha256.
Performance on Intel Skylake (Higher is better):
Interesting. But if something different than SHA256 were implemented, it wouldn’t be compatible with existing backups.
I don’t believe Black2b produces equivalent hashes as SHA3 - it seems to be a different algorithm.
Regarding security - that’s not the purpose of hashing in Duplicati. It’s not used for any cryptographic purposes. I’m not an expert by any means, but I believe the main goals are to use a proven hashing function with good performance while minimizing chances of collision. There’s an inherent conflict here - larger hash size (greater number of bits) reduces chance of collision but increases computation power.
There’s also BLAKE2bp which is designed for multi-threading processors for parallel processing.
The question always is what’s the bottleneck for processing. It’s really hard to give generic answer because there’s none. Sometimes parallelism improves performance, and sometimes it totally destroys it.
But from the standpoint where security doesn’t matter, it’s just enough to have unique hashes, the BLAKE2 would be a great option, in my opinion.
In some environments, the hashing performance is the bottleneck of backup processing and in these situations it would naturally improve performance. But sure, of course it would be incompatible with existing backups. If implemented in parallel, it just would lead to churn of the data, where old copies are pruned when those expire and replaced with new blocks using BLAKE2b.
Btw. When I mentioned support for different kind of platforms, BLAKE2s is optimial for 32 bit platforms. It’s just like all platforms do not have AES-NI instruction set.