Slow backup in version v2.0.3.10 downgrade to version 2.0.3.5

1

Hello
Testing the new version of duplicati v2.0.3.10 with a volume of almost a tera, I already had this backup running in version 2.0.3.5.
After updating the backup was extremely slow, the same problem of slowing persists.
I have 95% of my backups in version 2.0.3.5 these work very well.
Only on my machine do I have the latest version.
I believe that a solution to this problem will soon come out.
Thank you
Anderson

2

Just to confirm, from my initial tests of 2.0.3.10, it appears that my backups are now running just over 10x slower. The same backup job which was previously taking 5 or 6 minutes when there weren’t many changes / new files, is now apparently taking around 70 minutes.

3

In case anyone didn’t spot it (page was down for awhile), information and troubleshooting can be found here.

1 Like
4

Just for the sake of more data points. My Ubuntu server seems to be performing a decent amount faster at about 5 min to process 200GB. It’s previously which has taken anywhere from 6 to 10 minutes.

5

Can you confirm what OS this was on? I had the same experience on my Windows 8.1 machine. But on my Windows 10 laptop, backups are now faster.

6

Hello
I am using Windows Server 2008 R2

1 Like
7

I upgraded to v2-0-3-10 on Windows 2012 R2 before it was with the V2-0-3-6 version the backup took an average of 06 minutes, after updating the same is still doing, this without having any file changes.
On my Windows 10 machine I did not feel any difference, I’m using v2-0-3-10.

8

After a number of reports of this slowdown and possible mitigations, I will put up a v2.0.3.11 with a faster query as default.

This version will support a number of alternate queries, which can be used to further optimize the execution time for backups.

For most users this should mean that 2.0.3.11 should be significantly faster than 2.0.3.10, and hopefully faster than 2.0.3.9 as well.

For users that want to help out with getting things even faster, there is an environment variable that can be used to choose a different query to do the lookup.

To use this, you need to set the environment variable TEST_QUERY_VERSION to one of the following values:

  • TEST_QUERY_VERSION=1
    Use the original query used until Duplicati v2.0.3.9

  • TEST_QUERY_VERSION=2
    The default version used in v2.0.3.11, reported to be the fastest in v2.0.3.10

  • TEST_QUERY_VERSION=3
    An unverified contender to be the fastest query, proposed by @prez

  • TEST_QUERY_VERSION=4
    The query used in v2.0.3.10, but with an added optimization

If you set the environment variable correctly, you will get a warning after running a backup with the message Using performance test query version ___ as the TEST_QUERY_VERSION environment variable is set.

After testing each version, make sure to check that “OpenedFiles” is not equal or close to “ExaminedFiles” as that indicates that the query fails and each file is scanned for changes.

1 Like
9

I should mention that I don’t know how my query will effect the examined vs. opened question. My changes to the query were merely to take the default you had before (from 2.0.3.10), and reorganize it so it was a little more efficient. If your original query was wrong, so is mine, as I didn’t change it’s functionality, just hopefully how fast it executes :smiley:

2 Likes