Recently upgraded from I believe 2.0.3.3_beta to 2.0.4.5_beta. Everything appears to be working, however it’s MUCH slower than the previous version.
I have 4 different backup sets, on 2.0.3.3 the backup times were 00:13:34, 00:11:45, 00:03:22, and 00:45:19 respectively.
On 2.0.4.5 the times are 00:20:12 (not too bad), 02:01:42 (almost 2 hours longer), 00:38:33 (35 minutes longer), and 04:26:16 (3.5 hours longer) respectively.
An additional difference, after the update I enabled the default filters on each backup set.
Any ideas why the backup times would be so much longer now?
I have just finished the second round of backups under 2.0.3.3. Here’s the rough stats:
Initial Backup of 3.7 TB: 15 days 11 hours
Scan of this dataset of 3.7 TB (no data change): 50 mins
2nd Round of additional 900 GB (on top of the 3.7 TB): 4 days 20 hours
Scan of this dataset of 4.6 TB (no data change): 52 mins
After updating to 2.0.4.5:
Scan of this dataset of 4.6 TB (no data change): 1 hour 32 mins
I do not have filters or exclusions enabled at all; this is done on a Debian server powered with a Xeon E3-1245 (same as Sandy Bridge i7 2700)
This part seems to be taking longer than it should:
Dec 7, 2018 3:16 PM: Starting - ExecuteScalarInt64: SELECT COUNT(*) FROM “Block” WHERE “Size” > 102400
Dec 7, 2018 3:16 PM: ExecuteReader: SELECT “Key”, “Value” FROM “Configuration” took 0:00:00:00.000
Dec 7, 2018 3:16 PM: Starting - ExecuteReader: SELECT “Key”, “Value” FROM “Configuration”
Dec 7, 2018 3:16 PM: ExecuteReader: SELECT “Key”, “Value” FROM “Configuration” took 0:00:00:00.000
Dec 7, 2018 3:16 PM: Starting - ExecuteReader: SELECT “Key”, “Value” FROM “Configuration”
Dec 7, 2018 3:16 PM: ExecuteScalarInt64: INSERT INTO “Operation” (“Description”, “Timestamp”) VALUES (“Backup”, 1544166987); SELECT last_insert_rowid(); took 0:00:00:00.064
Dec 7, 2018 3:16 PM: Starting - ExecuteScalarInt64: INSERT INTO “Operation” (“Description”, “Timestamp”) VALUES (“Backup”, 1544166987); SELECT last_insert_rowid();
Dec 7, 2018 3:16 PM: Starting - Running Backup
Dec 7, 2018 3:16 PM: The operation Backup has started
Addendum:
A reboot of the server made it worse; the status bar and logs are stuck at the “Starting Backup” phase and there is no way to stop the backup.
I am rebooting the server again and then recreating the database from scratch to see if it helps. I have backed up the 2.0.3.3 database and configs already so if this still does not work out I’ll downgrade to 2.0.3.3
Is it correct that it is not possible to easily downgrade from 2.0.4.5_beta to 2.0.3.3_beta in order to avoid the super slow backups until this can be fixed? (I did not create a backup of the database before upgrading)
I’m running into this issue as well, I actually thought my install was somehow broken because I’m having processes that used to take 1-2 minutes (its only a 60gb backup set on a fairy fast SSD with 24 cpus available and mostly idle) take over 24hr with no foreseeable progress being made.
Things such as verifying files that used to take a few seconds to run now take 45min-1hr+ just to start…?
I’m not sure what everyone is talking about with the default filters, on my backup sets I have every filter disabled except for 3 file/folder name exclusions and I don’t see any other default options set, etc… if someone could explain what they mean by default filters, I’d gladly try and disable them (if enabled) to see if the issue goes away.
For now I kinda am just stuck in the water with 3 systems that now can’t seem to backup due to how long it takes to do so and I’m mystified at why after the upgrade…
For the record this is under Windows 10 right now.
It looks like the existence of a regex filter can severely impact performance. Of the built in excludes, the “Cache Files” group (which is included in the “Default excludes” group) has a regex filter. One thing you can try is to disable the “Default excludes” group, and include all other groups except the “Cache Files” group. This will hopefully improve performance at the expense of additional files being added to the backup.
I can confirm that I don’t have any default excludes enabled based on that screenshot and I have no regex either, only 3 “exclude folders whose names contains”, but its also only around 160,000 files in total as well.
I’m tempted to try the 2.0.4.6 build to see if the issue is resolved there…?
If you can survive with the slower backup a little while longer, consider holding off on trying 2.0.4.6 a bit as some people have reported issues with installing it (I haven’t confirmed those) and the UI in IE (confirmed and fixes already in testing) after updating.
I am having this problem also, but the effect is not the same for all backups.
I have 3 remote locations all backing up back to the home office. only one location saw a massive increase in time for backups. it increased from ~4 hours to ~11 hours. The other sites went from ~2 hours to ~5 hours. The size of the data is close to the same at all the sites, but the one with the massive increase in time has almost 3x the number of files. So, even before the 2.0.4.5 release, filtering more files took more time. It appears that the regex problem magnifies this greatly.
When concurrent processing was introduced in version 2.0.3.6, several users had performance issues related to file verification, which may be related to the size of the database. More details here. Some of these issues were later addressed, but apparently the issue still exists for some users.
Looks like this is still not fixed in 2.0.4.23 beta! At least here for Mac user, unacceptable performance hit when exclude filter group items are selected.
This is a cumulative release for more than 6 months of hard work.
Most of these are speedup improvements, error handling and general quality fixes.
and above hints at long intervals between beta releases. Fixes don’t help much until they’re released. Above Experimental didn’t make beta due to an FTP bug and test need. Someday new Beta will ship. Release process improvements are also being proposed to see if the release rate can be increased.
