Restore from large backup

[bccvs]

Hi,
Currently having a 671GB database, 54K files, 300 versions.

Restoring takes forever, seems like hanging at “recreating database” at about 90%.

Any ideas ? Will this finish eventually ?

Thanks
best regards
Christophe

[drwtsn32]

What version of Duplicati are you using? There is a bug in some versions that causes a database recreation to take much longer than should be required.

[bccvs]

Hi,

Currently using 2.0.4.23_beta_2019-07-14

[drwtsn32]

Apologies, not sure how I missed this reply until just now. Says you sent it 6 days ago but I was only notified recently!

Anyway - it is possible you are affected by a database recreation bug that causes it to take far longer than it’s supposed to. This bug has been fixed but the fix hasn’t made it into the beta channel yet.

If you are willing to try a canary version, you may have better luck. Be warned that the current canary release does have certain OTHER flaws not present in the beta version. Also, you won’t be able to go back to the beta version (easily) because the latest canary upgrades your databases. You’ll have to wait for the next beta release to switch back to that channel.

Similar issue was discussed recently in this thread:

How can you manually uncompress/extract backup fies

Wow what a huge difference. I upgraded to version 2.0.4.30_canary_2019-09-20 and it took hardly no time lol. What a huge difference that made. Thank you. One other thing. Not related to this but I did a restore one day and it took forever. I looked at the log and it kept displaying file permission errors. I then ran Duplicati as administrator and it did not have any file permission errors and it restored the files rather quickly. I think Duplicati should always be run as administrator.

Canary worked great for this user. Note the posts that talk about the flaws in the canary release so there is some added risk.

[bccvs]

Thanks,

I already replied yesterday,

[Hi,

Updated to Duplicati - 2.0.4.30_canary_2019-09-20,

Starting restore,

Recreating database seems stuck at 90% for a couple of days,

Any ideas ?

Best regards

Christophe

]

Which canary version to use ?

Thanks,

Best regards

Christophe

[drwtsn32]

@bccvs

Which canary version to use ?

Per this post (in the thread I linked):

How can you manually uncompress/extract backup fies

Empty source file can make Recreate download all dblock files fruitlessly with huge delay #3747 was fixed in 2.0.4.18. Unfortunately dblock put retry corrupts dindex, using old dblock name for index – canary regression #3932 seems to have been added in 2.0.4.16, and the purge issue mentioned (also affects repair) in 2.0.4.29. So if you got your files back, the Duplicati version to resume backup on is a question, given issues in canary.

You need at least version 2.0.4.18 to get the database recreation fix. Personally I would use the latest 2.0.4.30, but do not use the purge command. (I’m guessing its usage is uncommon.)

[ts678]

To be clear, “added in 2.0.4.16” means “still present in 2.0.4.30” on which backup corruption bug was filed., therefore “do not use the purge command” also needs “do not have any upload errors, e.g. due to Internet”, which is beyond your control. It should be fine for restores, AFAIK, but I’m kind of nervous about backups…