I’m new to duplicati so please be patient with me here. While i was setting up my Duplicati instance i noticed the advanced option --file-read-buffer-size=0kB
According to the online documentation “Use this size to control how many bytes a read from a file before processing.”
Does anybody have a bit more information about this feature?
For instance, when it says “Before processing”, i assume it makes reference to the actual write to the target folder, correct?
Also, where is this buffer being saved? on RAM? on a temp folder in the system drive?
Lastly, what is a sensible size to specify here? 500 MB for instance?
C:\ProgramData\Duplicati\duplicati-2.0.5.114_canary_2021-03-10>Duplicati.CommandLine.exe help file-read-buffer-size
--file-read-buffer-size (Size): Size of the file read buffer
[DEPRECATED]: The "file-read-buffer-size" option is no longer used and has
been deprecated.
Use this size to control how many bytes are read from a file before
processing
* default value: 0kb
but the manual and the online documentation built into the Duplicati web UI don’t speak to deprecations.
This is probably a bad thing, because as time passes, there are more and more of them being created.
I think the processing is (or I should say, was) on the input side, after the read whose data gets hashed in memory to see if it’s been backed up before (deduplication). If you care to study, 2.0.3.3 code starts here.