Very slow database rebuild

Unfortunately when dblocks are involved, it’s a pretty slow process. I believe it is may be due to sqlite performance limitations, or possibly Duplicati is lacking some sqlite optimizations. It definitely needs more work.

Normally a database can be rebuilt without dblocks, or at least without too many of them. I’m curious to see what your Live Log says about the number of dblocks that need processing.

This past weekend I was forced to do a ‘refresh’ on my Windows 10 install. This of course wiped out my Duplicati databases. I’m trying to rebuild one, but it appears to be doing nothing for the past 36 hours. Like others here, it went quickly up to a point and then has done nothing since. And when I mean nothing, I mean the process time in Task Manager has remained at 01:07:19.

This is a rather large backup set of all my photos off my dSLR. To resend this all up to the cloud would take all month.

Is there not a backup of the database stored along side the backup? How does a restore work? Can this database be restored from there somehow?

Edit: Additionally “verbose” logging has not displayed anything in that period as well.

What do you mean a ‘refresh’? Did you reinstall Windows? Or did it do a semi annual update?

If the latter and you were running Duplicati as a service, we can probably find your database inside C:\Windows.old.

The database is not backed up to remote storage. But normally it can be rebuilt from remote storage without too much difficulty.

What version of Duplicati are you running?

Similar, but an inplace reinstall basically, the “Reset this PC” option.

Thank you very much for pointing that out. The databases I need are in there. I always thought that was just the previous copy of Windows, never thought it’d include the Users directory (I never looked, never needed to).

Thanks so much!

Edit: Oh yeah, I’m running 2.0.5.1_beta_2020-01-18
Edit2: And also yes, the semi-annual update broke my machine, which is what caused me to do the ‘refresh’/reset.

I recommend you move the databases to a different location. C:\Windows is risky, and hopefully Duplicati will be improved in the future so it stops using that location (when running as a service).

It wasn’t in C:\Windows it was in C:\Windows.old\Users\user\AppData\Local\Duplicati\ if I remember correctly. The Users folder was in the C:\Windows.old directory and that’s where I recovered my databases from.

I had no idea Windows made a backup of the \Users\ directory in the \Windows.old\ directory.

And as an aside, I see that my database directory is ~40 GB in size.

Do I need to keep all the files starting with backup and ending in .sqlite. They seem years old.

Ex: backup 20181205034555.sqlite

Removing those would free up 26 GB.

The ‘backup*.sqlite’ files don’t need to be retained.

Where is your active database currently located? It’s not C:\Windows…?

Active is the default location C:\Users\user\AppData\Local\Duplicati\