Very high and long activity on local b during (incr.) backups

Hi, I have tested duplicati2 for some weeks and was at the step to put it in production mode
for my personal backups. The final test was a backup of a local drive with ~160000 files/dirs in aprox. 600 GB to a 2nd local high speed drive.
The first (full) backup run was OK (few hours) and then I tried to do the 2nd one -
with just a few files changed (~10).
Astonishingly this took a long time - more than 20 mins. .
Doing some investigations I saw in the Win10 resource montor that almost the whole time is spent
reading the local database (sqlite) file - reading it at a rate of 500k/sec. The local db has a size of ~1GB.
One of the most important reasons to change over from dup1 to dup2 is the new retention
logic avoiding new full backups - but if the normal ‘daily’ backup now needs 20 mins. for 10 small changed files. Old duplicati1 did this in 3 mins. .

I also tried moveing the local db to a very fast ssd - same result.

I am running the backup all 2 hours - so 3 mins. are OK (dup1) but the 20 mins. with high disc
activity is no fun.

Any ideas ?

Welcome to the forum!

I’m not entirely sure what is going on, but maybe try (temporarily) setting the --no-auto-compact option. See if that makes a difference on how much time your backups take.

When I had a very large database, I found that the database queries to decide if compaction should happen would take several minutes to complete.

Also, are you using the --auto-vacuum option? For large databases this can add several minutes to the end of the backup job.

1 Like

Using canary solved this topic - the follow up backups (incr. …) are now done in 6 mins. - :-).
Thanks and happy new year.