Duplicati restore is very slow

Nobody’s saying they have an SSD, so random access time on hard drive may be one speed limiter.

no-local-blocks

Duplicati will attempt to use data from source files to minimize the amount of downloaded data. Use this option to skip this optimization and only use remote data.

The assumption is that your local disk is faster than your remote (which might also have egress fees).

Restore also fixes up small problems (such as a bad version) block by block instead of with whole file.

The backup process explained in the manual tries to describe this in non-technical language involving bricks in different shapes and colors, stored in small bags. I think the restore equivalent is to grab one bag at a time, and distribute any brick that a file needs, wherever the file is (thus doing random writes).

The development focus has been on making things work. Backups got a design change to work faster. Restore hasn’t had that, due to too many priorities and too few volunteers, so it’s still rather sequential.

I’m talking mainly about the multi-terabyte original post. The small 42 gigabyte issue might be different.

Features

Incremental backups
Duplicati performs a full backup initially. Afterwards, Duplicati updates the initial backup by adding the changed data only. That means, if only tiny parts of a huge file have changed, only those tiny parts are added to the backup. This saves time and space and the backup size usually grows slowly.

So if you meant incremental works well, you might have forgotten that the initial backup took it awhile. Description sounds like a full restore, but I’m not sure. Another slow-down to even starting restore is if database recreate is needed due to system loss. That takes awhile, and damaged backup gets worse.

Restoring files if your Duplicati installation is lost is worth testing with at least a small restore sometime, however a full database recreate (maybe save the old database in case new has issues) is also useful.

Choosing sizes in Duplicati has some advice. One frequent performance limiter with backups over 100 GB is that The block size is only 100 KB by default, and slows down due to too many blocks to handle.

maybe will happen someday to allow more graceful handling of at least 1 TB source backups. The loss from that is that block level deduplication saves less space, but some people care more about speeds. Going overboard (beyond simple blocksize scaling up to stay at around a million blocks) could be tried.

blocksize

The block size determines how files are fragmented. Choosing a large value will cause a larger overhead on file changes, choosing a small value will cause a large overhead on storage of file lists. Note that the value cannot be changed after remote files are created.