Hangs while trying to "Re-creating missing index file"

#1

Hi,

My backup have been failing for some time now, and today I finally had the time to look into it. It seems to me like it hangs when it is in the process of re-creating a missing index file. Can someone suggest something to test, to make it continue.

I’m now on v.2.0.4.16, but I used to be on the current beta until tonight. I believe I had the same issue.
Destination is Jottacloud.

The log says the following:

    1. april 2019 kl. 00:06: Re-creating missing index file for duplicati-b75c03ab613984f958c14d9cabd372ce5.dblock.zip.aes
    1. april 2019 kl. 00:05: The operation Backup has started

If I look at the log in verbose mode, it loops through all my files and then stops. The log entries look like this:

    1. april 2019 kl. 00:13: Including path as no filters matched: E:\Filenames
    1. april 2019 kl. 00:12: Including path as no filters matched: E:\Filenames

It stays like this for a while, with no indication of anything happening in any of the logs I can find. But the Duplicati.Server process still uses ~30% CPU.

Any suggestions to where I can continue my debugging?

Thanks,
Erik

1 Like

#2

I have seen this taking a very very very long time. Hold on or move it to a more powerful machine if possible…

0 Likes

#3

Would that be hours or days for a 2TB backup set? :slight_smile:

0 Likes

#4

It depends how many index files are missing. Once you know that and the time it took for once you can start estimating…

0 Likes

#5

Thanks! Fortunately the backup job managed to complete, and it took 3,5 hours to complete for my 2TB backup. It used to take only 30 mins or so.

I was hoping it would be quicker the next time, but unfortunately, it was, it is still running after almost 4 hours. Is there any suggestions to how I can make it quicker again? :blush:

Thanks in advance!

0 Likes

#6

The more versions you have, the longer it takes. Try to reduce by applying a retention policy.

You might also want to consider splitting it in several jobs. Eg. One that has all data but runs once in a month, and one that has only data from 2019 ( example, depends on your data), and runs every day.

0 Likes

#7

Do you happen to know if the GUI says “Verifying backend data” when the detailed log underneath is saying “Re-creating missing index file”? That was my speculation at below, where I also mention a 2.0.4.17 fix you might want, because it might give you fewer missing index files to have to deal with. Might have other ideas.

Verifying Backend Data

0 Likes

#8

Thanks for your input!

I thought it was done with the re-creation of missing index files earlier this week, but I was wrong. It turns out the backup completed only when I disable the pre backup verification. When I turned it off again, it have now been going on re-creating index files for the last 6 days, continuously.

The weird thing, and the reason I believed it to be a hanged process in the first place, is that it gives absolutely no indication that anything is happening for 3,5 hours. Then it completes an index file and then starts on the next. Since I didn’t expect this (it hasn’t happened to me before), I thought when nothing happened for a couple of hours, it was broken. Not so! It just takes 3-4 hours per index.

I have now upgraded to 2.0.4.17 in the hope that it will fix this index-issue. :slight_smile:

I do have a retention policy in place, but I see that I might have to split the backup in several jobs. It’s just very convenient to have it all in one place!

0 Likes