Auto-run repair if files missing?

Support
1

Hi,

Problem

I’m one of many, which facing the problem where Duplicati reports that x amount of files is missing, like:

...
2021-02-20 01:33:06 +01 - [Error-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteFiles]: Found 4 files that are missing from the remote storage, please run repair
2021-02-20 01:33:06 +01 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
Duplicati.Library.Interface.UserInformationException: Found 4 files that are missing from the remote storage, please run repair
  at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend,
...

Everytime the repair fixes this. Here’s the “Repair” report for the latest repair-job:

MainOperation: Repair
ParsedResult: Success
Version: 2.0.5.1 (2.0.5.1_beta_2020-01-18)
EndTime: 2/20/2021 10:29:37 AM (1613813377)
BeginTime: 2/20/2021 10:29:13 AM (1613813353)
Duration: 00:00:23.8561070
MessagesActualLength: 11
WarningsActualLength: 0
ErrorsActualLength: 0
LimitedMessages: [
    2021-02-20 10:29:13 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started,
    2021-02-20 10:29:20 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
    2021-02-20 10:29:35 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (2.85 KB),
    2021-02-20 10:29:36 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:36 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:36 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:36 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:36 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:37 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-<removed>.dindex.zip.aes (541 bytes),
    2021-02-20 10:29:37 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-<removed>.dindex.zip.aes (541 bytes),
...
]
LimitedWarnings: []
LimitedErrors: []

...

MainOperation: Repair
ParsedResult: Success
Version: 2.0.5.1 (2.0.5.1_beta_2020-01-18)
EndTime: 2/20/2021 10:29:53 AM (1613813393)
BeginTime: 2/20/2021 10:29:37 AM (1613813377)
Duration: 00:00:16.0831230
MessagesActualLength: 4
WarningsActualLength: 0
ErrorsActualLength: 0
LimitedMessages: [
    2021-02-20 10:29:37 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started,
    2021-02-20 10:29:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
    2021-02-20 10:29:53 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (2.86 KB),
    2021-02-20 10:29:53 +01 - [Information-Duplicati.Library.Main.Operation.RepairHandler-DatabaseIsSynchronized]: Destination and database are synchronized, not making any changes
]
LimitedWarnings: []
LimitedErrors: []

Can I force a single repair attempt, after a failed backup-job?

I’ve tried and search for a good solution to handle this automatically, but couldn’t find a way to force a single repair attempt, after a failed backup.

Are there a hidden feature, not mentioned in the Advanced Page, or should I handle this issue, in another way?

Please let me know how it’s best to handle this, but without having me to sign in and press the “Repair”-button.

System

Version: 2.0.5.1_beta_2020-01-18
OS: CentOS
Remote Backup Solution: Microsoft SharePoint v2
Frequency of error: Once every month or so, so it’s not that big of a deal - but still annoying.

2

There might be a clever way to script a single repair attempt by using pre and post backup scripts, checking operation type, and backup result code, etc.

But I think a better approach is to figure out why this is happening to you in the first place! It is worrisome to me that files are disappearing on the back end.

3

That’s a good point.

To turn it around: How would I spot it? The “backend files”, are that my source-files I’m backing up, or is it the files on the backend-end, in this case Microsoft SharePoint v2?

I’ve spotted the issues a few times as mentioned, but the fix works everytime. Therefore I’m wondering if it’s a bug with the SharePoint integration, or so?

I would like to see an option like that, and quite a few people have reported the same issue, if I search on Google. I’m not sure it they are seeing the same issues over and over again, but I sounds a bit like that.

4

Back-end are the files Duplicati places in the “destination.” Duplicati itself can do a good job catching this problem, as it looks at the file listing before every backup.

I don’t doubt that you can “fix” this by running repair, but the real question is why these files are going missing in the first place. If it were a bug in the SharePoint integration itself, I think we’d have tons of people reporting this problem in the forum.

I personally do not use the SP/OneDrive back end so have no experience with it.

5

Any pattern before? Duplicati checks for missing files at start of a backup, so issue might be on prior one.
You can look at prior results using both the usual job logs and the server log About → Show log → Stored.

Anything other than a clean run might have left an issue. You can also look for patterns in your Repair logs.
The one you show looks like it was replacing some surprisingly small dindex files, or are they all that size? You probably have some GUI or other tool to look in SharePoint. Also, is it always dindex files that are lost?

It would help to know the history of the missing files. Your database has this, if you’re willing to open to look, or post a sanitized version with Create bug report, or run a long-term log-file=<path> log-file-log-level=retry.

On that last note, looking in the Complete log at RetryAttempts might be useful, but they are not visible until they turn into an error after number-of-retries exhausts, and I already asked about issues in prior runs.

6

Hi @ts678

No, sadly no patterns at all. But I’ll keeps my eyes on anything. I’ve moved the server to another VPS-instance, and I’m not sure that it’s 100% fixed - but I’ve not seen the issue for the last two months.

I’ll post here again, if that’s relevant.

Thank you for the answer, anyway :slight_smile: !.

7

I’m not going to say it’s best (especially since it’s causing issues for someone), but we’ve noticed that the auto-cleanup option looks like it does VerifyRemoteList, then runs Repair if it finds it wrong. This is before backup, not after. One current problem report began in forum here and is now in GitHub as an issue here.

Although I mention that, I’m not encouraging its use right now, especially with some possible issues in it…

Thank you. If it fails again, we can resume examining things using logs and other tools mentioned above.

1 Like