Clarify volume size

Hello, first of all, thank you for your contribution. I have a few questions. First, why does a volume size larger than 1GB cause file processing to slow down? Also, when I set the volume size to 4GB, I noticed that the temporary files it generated took up a lot of space, and then it was reduced to 40GB, which is 10 volumes. Could you please explain the general workflow? What is the maximum size of the temporary files it generates, and approximately how many times the volume size is it? Furthermore, I set the asynchronous-upload-limit to 4, but it still generated 40GB of temporary files. Is this not supported by the software now, or is the limit not on the number of temporary volumes? Thank you for your answers.

Hi @WhoCare, welcome to the forum :waving_hand:

Essentially, blocks can be scattered among volumes, say you have a file with 5 blocks of 1 MiB each and a 4GiB volume size. Ideally, all 5 blocks are in the same volume, so restoring needs to download 4GiB to restore 5 MiB for a partial restore.

But, worst case is that all blocks are scattered in different volumes, causing 4GiB*5 = 20GiB of files to be downloaded to restore the same 5MiB.

Also, for the new restore flow, it tries to re-use the downloaded files, meaning that they stick around in the temporary storage for longer, so you may either need a big temporary storage or have repeated downloads of the same (large) files.

This is described here:

Essentially, you get:

temporary-files = 
  asynchronous-upload-limit + concurrency-compressors