Duplicati 2.0.4.5_beta much slower than 2.0.3.3_beta

Recently upgraded from I believe 2.0.3.3_beta to 2.0.4.5_beta. Everything appears to be working, however it’s MUCH slower than the previous version.

I have 4 different backup sets, on 2.0.3.3 the backup times were 00:13:34, 00:11:45, 00:03:22, and 00:45:19 respectively.

On 2.0.4.5 the times are 00:20:12 (not too bad), 02:01:42 (almost 2 hours longer), 00:38:33 (35 minutes longer), and 04:26:16 (3.5 hours longer) respectively.

An additional difference, after the update I enabled the default filters on each backup set.

Any ideas why the backup times would be so much longer now?

There might be a performance issue with the default filters:

Can you try temporarily disabling the default filters to see if that improves anything?

Thanks for your reply!

Sure thing. I was going to before I posted but the 4 hour backup was running. Decided to just cancel it.

Ran the shortest backup just now with the default filter disabled and it’s back to 00:03:27. Looks like that’s the culprit.

1 Like

Thanks for confirming. Hopefully we’ll be able to figure out a way to address the performance issues with the regex filters.

I have just finished the second round of backups under 2.0.3.3. Here’s the rough stats:
Initial Backup of 3.7 TB: 15 days 11 hours
Scan of this dataset of 3.7 TB (no data change): 50 mins
2nd Round of additional 900 GB (on top of the 3.7 TB): 4 days 20 hours
Scan of this dataset of 4.6 TB (no data change): 52 mins

After updating to 2.0.4.5:
Scan of this dataset of 4.6 TB (no data change): 1 hour 32 mins

I do not have filters or exclusions enabled at all; this is done on a Debian server powered with a Xeon E3-1245 (same as Sandy Bridge i7 2700)

This part seems to be taking longer than it should:

  • Dec 7, 2018 3:16 PM: Starting - ExecuteScalarInt64: SELECT COUNT(*) FROM “Block” WHERE “Size” > 102400

  • Dec 7, 2018 3:16 PM: ExecuteReader: SELECT “Key”, “Value” FROM “Configuration” took 0:00:00:00.000

  • Dec 7, 2018 3:16 PM: Starting - ExecuteReader: SELECT “Key”, “Value” FROM “Configuration”

  • Dec 7, 2018 3:16 PM: ExecuteReader: SELECT “Key”, “Value” FROM “Configuration” took 0:00:00:00.000

  • Dec 7, 2018 3:16 PM: Starting - ExecuteReader: SELECT “Key”, “Value” FROM “Configuration”

  • Dec 7, 2018 3:16 PM: ExecuteScalarInt64: INSERT INTO “Operation” (“Description”, “Timestamp”) VALUES (“Backup”, 1544166987); SELECT last_insert_rowid(); took 0:00:00:00.064

  • Dec 7, 2018 3:16 PM: Starting - ExecuteScalarInt64: INSERT INTO “Operation” (“Description”, “Timestamp”) VALUES (“Backup”, 1544166987); SELECT last_insert_rowid();

  • Dec 7, 2018 3:16 PM: Starting - Running Backup

  • Dec 7, 2018 3:16 PM: The operation Backup has started

Addendum:
A reboot of the server made it worse; the status bar and logs are stuck at the “Starting Backup” phase and there is no way to stop the backup.

I am rebooting the server again and then recreating the database from scratch to see if it helps. I have backed up the 2.0.3.3 database and configs already so if this still does not work out I’ll downgrade to 2.0.3.3

Is it correct that it is not possible to easily downgrade from 2.0.4.5_beta to 2.0.3.3_beta in order to avoid the super slow backups until this can be fixed? (I did not create a backup of the database before upgrading)

I’m running into this issue as well, I actually thought my install was somehow broken because I’m having processes that used to take 1-2 minutes (its only a 60gb backup set on a fairy fast SSD with 24 cpus available and mostly idle) take over 24hr with no foreseeable progress being made.

Things such as verifying files that used to take a few seconds to run now take 45min-1hr+ just to start…?

I’m not sure what everyone is talking about with the default filters, on my backup sets I have every filter disabled except for 3 file/folder name exclusions and I don’t see any other default options set, etc… if someone could explain what they mean by default filters, I’d gladly try and disable them (if enabled) to see if the issue goes away.

For now I kinda am just stuck in the water with 3 systems that now can’t seem to backup due to how long it takes to do so and I’m mystified at why after the upgrade…

For the record this is under Windows 10 right now.

Unfortunately yes, that is correct.

See if you’ve got anything like in this image:

Note that if you do and you disable it then your backup is likely to start including a bunch of guess it used to ignore so might still be slow…

It looks like the existence of a regex filter can severely impact performance. Of the built in excludes, the “Cache Files” group (which is included in the “Default excludes” group) has a regex filter. One thing you can try is to disable the “Default excludes” group, and include all other groups except the “Cache Files” group. This will hopefully improve performance at the expense of additional files being added to the backup.

I can confirm that I don’t have any default excludes enabled based on that screenshot and I have no regex either, only 3 “exclude folders whose names contains”, but its also only around 160,000 files in total as well.

I’m tempted to try the 2.0.4.6 build to see if the issue is resolved there…?

A major bummer

What version were you using previously?

Would have been following the default auto update channel, so 2.0.3.3 previously.

If you can survive with the slower backup a little while longer, consider holding off on trying 2.0.4.6 a bit as some people have reported issues with installing it (I haven’t confirmed those) and the UI in IE (confirmed and fixes already in testing) after updating.

I am having this problem also, but the effect is not the same for all backups.

I have 3 remote locations all backing up back to the home office. only one location saw a massive increase in time for backups. it increased from ~4 hours to ~11 hours. The other sites went from ~2 hours to ~5 hours. The size of the data is close to the same at all the sites, but the one with the massive increase in time has almost 3x the number of files. So, even before the 2.0.4.5 release, filtering more files took more time. It appears that the regex problem magnifies this greatly.

That would make sense as the regexes would have to be checked more times.

Thanks for sharing your comparison numbers!

I also upgraded from 2.0.3.3 to 2.0.4.5 on a linux server. Performance drop is quite large. Before upgrade 1.30h after 2.30h.

I wonder if there are several potential things going on:

  1. The slow regex filters discussed here.
  2. When concurrent processing was introduced in version 2.0.3.6, several users had performance issues related to file verification, which may be related to the size of the database. More details here. Some of these issues were later addressed, but apparently the issue still exists for some users.
1 Like