Found 1 remote files that are not recorded in local storage

Duplicati had been running for months without incident. My backup drive is a SATA drive in a removable drive bay. The drive bay (not the hard drive) had to be replaced recently, and so backups were not performed for about a week.

Yesterday the new drive bay arrived, and I installed it, inserting the same hard drive.

I have 5 scheduled backups configured. Three of them ran last night. Two were successful. The third is reporting the error:

Found 1 remote files that are not recorded in local storage, please run repair

I click “Repair,” and the UI acts as if it is doing something, but I see no results, and the error remains. I click Repair repeatedly, with the same result. I see no errors in the log (attached), and no indication of what the file is.

Duplicati - 2.0.6.3_beta_2021-06-17
Linux Mint 20.3
DuplicatiLog.zip (1.9 KB)

What further info can I provide?

Today it reported 13 files. It’s odd that when I Show Logs, there’s no mention of the errors:

So I don’t know if there’s really a problem with files, or if it’s just a spurious error.

I don’t know how this in itself would cause issues. Was this job affected by whatever forced the repair?
Ideally a running Duplicati backup runs to completion. Interruptions usually resume OK, but not always.

I get a job log showing the Extra dindex file I dummied up. We don’t know your files, but is there a log?

image

“2022-02-11 15:18:57 -05 - [Information-Duplicati.Library.Main.Operation.RepairHandler-AcceptNewIndexFile]: Accepting new index file duplicati-i571d9bd1345a4a6c8d15771ca42b1c86.dindex.zip”

More direct (and it won’t attempt repair) is to open another tab to About → Show log → Live → Warning
The Verify files button will do a similar file list analysis to what backup does, but won’t try a backup.

You can also look at the backup job logs to see if a Compact ran (your posted log shows one), because sometimes Compact does the wrong thing, and is more likely to do this if interrupted in middle of things.

History is likely vague, but if issue continues you can set up log-file=<path> log-file-log-level=information which should provide a larger (but not huge – there are huger levels) history giving the actual file names.

From About → Show Log → Live → Warning:

Feb 12, 2022 7:53 AM: The operation Repair has failed with error: The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-b5d3257245f4a4d77af6d6e10e65016d1.dblock.zip, duplicati-be869695489274668b5e988073be72ec9.dblock.zip, duplicati-be16be1a843b343ed9bb37945f98dd957.dblock.zip, duplicati-bf118b8b56b8d4c799cfd30714e8a7bdd.dblock.zip, duplicati-be85ab3dc8e52472380626bb2afffdca9.dblock.zip, duplicati-b1c8d85f535bc4b23bc48c2db7918c555.dblock.zip

{“ClassName”:“Duplicati.Library.Interface.UserInformationException”,“Message”:“The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-b5d3257245f4a4d77af6d6e10e65016d1.dblock.zip, duplicati-be869695489274668b5e988073be72ec9.dblock.zip, duplicati-be16be1a843b343ed9bb37945f98dd957.dblock.zip, duplicati-bf118b8b56b8d4c799cfd30714e8a7bdd.dblock.zip, duplicati-be85ab3dc8e52472380626bb2afffdca9.dblock.zip, duplicati-b1c8d85f535bc4b23bc48c2db7918c555.dblock.zip”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at Duplicati.Library.Main.Operation.RepairHandler.RunRepairRemote () [0x008d8] in :0 \n at Duplicati.Library.Main.Operation.RepairHandler.Run (Duplicati.Library.Utility.IFilter filter) [0x0016d] in :0 \n at Duplicati.Library.Main.Controller+<>c__DisplayClass18_0.b__0 (Duplicati.Library.Main.RepairResults result) [0x0001c] in :0 \n at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action1[T] method) [0x0026f] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action1[T] method) [0x00007] in :0 \n at Duplicati.Library.Main.Controller.Repair (Duplicati.Library.Utility.IFilter filter) [0x0001a] in :0 \n at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x003ad] in <156011ea63b34859b4073abdbf0b1573>:0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”}

log-file= log-file-log-level=information

DuplicatiInfoLog.txt.zip (904 Bytes)

I ran the backup with the rebuild-missing-dblock-files option. Now it reports only 6 missing files, down from 13. I re-ran with that option, but no change. Then I ran the ‘purge’ command. It only reports the missing files and suggests running “Repair” (which does nothing).

What it’s talking about, as seen in the help (adjusting for a seemingly misleading purge word) is:

  --rebuild-missing-dblock-files (Boolean): Rebuild dblock files when missing
    If dblock files are missing from the destination, you can attempt to
    rebuild them using local source data. However, since the local data may
    have changed, it may not be possible to retrieve all the required data
    and the process may be slow. Use this option to attempt to rebuild
    missing dblock files.
    * default value: false

as described in About → Changelog

Removed automatic attempts to rebuild dblock files as it is slow and rarely finds all the missing pieces (can be enabled with --rebuild-missing-dblock-files).

I “think” it’s a Repair option, and repair is not automatic but you can do manual run if it still sounds worth it.

The PURGE-BROKEN-FILES command is described further in Recovering by purging files which purges selected files having data in the dblock files that are missing. How and when that happened is a question.

Possible best case is something went wrong somehow near the drive work, and these files only affect new backups. The job database actually could show when they first appeared, but you’d either need to look with DB Browser for SQLite, or post a link to a database bug report for someone else to see what they can see.

Maybe the first thing to run is The LIST-BROKEN-FILES command, maybe in an edited GUI Commandline. This will give some indication of what files got damage, then maybe that will influence what gets done next.

1 Like