Testing the new version of duplicati v18.104.22.168 with a volume of almost a tera, I already had this backup running in version 22.214.171.124.
After updating the backup was extremely slow, the same problem of slowing persists.
I have 95% of my backups in version 126.96.36.199 these work very well.
Only on my machine do I have the latest version.
I believe that a solution to this problem will soon come out.
Just to confirm, from my initial tests of 188.8.131.52, it appears that my backups are now running just over 10x slower. The same backup job which was previously taking 5 or 6 minutes when there weren’t many changes / new files, is now apparently taking around 70 minutes.
In case anyone didn’t spot it (page was down for awhile), information and troubleshooting can be found here.
Just for the sake of more data points. My Ubuntu server seems to be performing a decent amount faster at about 5 min to process 200GB. It’s previously which has taken anywhere from 6 to 10 minutes.
Can you confirm what OS this was on? I had the same experience on my Windows 8.1 machine. But on my Windows 10 laptop, backups are now faster.
I am using Windows Server 2008 R2
I upgraded to v2-0-3-10 on Windows 2012 R2 before it was with the V2-0-3-6 version the backup took an average of 06 minutes, after updating the same is still doing, this without having any file changes.
On my Windows 10 machine I did not feel any difference, I’m using v2-0-3-10.
After a number of reports of this slowdown and possible mitigations, I will put up a v184.108.40.206 with a faster query as default.
This version will support a number of alternate queries, which can be used to further optimize the execution time for backups.
For most users this should mean that 220.127.116.11 should be significantly faster than 18.104.22.168, and hopefully faster than 22.214.171.124 as well.
For users that want to help out with getting things even faster, there is an environment variable that can be used to choose a different query to do the lookup.
To use this, you need to set the environment variable
TEST_QUERY_VERSION to one of the following values:
Use the original query used until Duplicati v126.96.36.199
The default version used in v188.8.131.52, reported to be the fastest in v184.108.40.206
An unverified contender to be the fastest query, proposed by @prez
The query used in v220.127.116.11, but with an added optimization
If you set the environment variable correctly, you will get a warning after running a backup with the message
Using performance test query version ___ as the TEST_QUERY_VERSION environment variable is set.
After testing each version, make sure to check that “OpenedFiles” is not equal or close to “ExaminedFiles” as that indicates that the query fails and each file is scanned for changes.
I should mention that I don’t know how my query will effect the examined vs. opened question. My changes to the query were merely to take the default you had before (from 18.104.22.168), and reorganize it so it was a little more efficient. If your original query was wrong, so is mine, as I didn’t change it’s functionality, just hopefully how fast it executes