2.1.1.100 > Found 1 faulty index files, repairing now

Updated few days back and got this warning two days back and today. Yesterdays run gave no warning. Any ideas what this means?

Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 1 faulty index files, repairing now

We recently discovered that the compact feature in older Duplicati versions would sometimes write index files that were not complete. This is a subtle issue that has gone unnoticed for a long time, because it only manifests itself in very slow database recreates (no data loss, but waiting for hours or days can be problematic).

We baked the repair process into the test process, so whenever Duplicati tests some random files, it detects if they are affected by the issue, and if so, fixes it by uploading a correct index files.

The net result should be that your backups are fixed over time, and if you need to recreate a database it should as fast as possible.

Thx a lot for the fast clarification :+1: :smiley:

I’m getting a lot of these

Do you mean lots at one time, or lots of times? The latter is because file testing is sampled.

Options backup-test-samples and backup-test-percentage control it, but to force-test.

The TEST command can be run in GUI Commandline with new (and not yet in docs) option

  --full-remote-verification (Enumeration): Activate in-depth verification of files
    After a backup is completed, some (dblock, dindex, dlist) files from the remote
    backend are selected for verification. Use this option to turn on full
    verification, which will decrypt the files and examine the insides of each
    volume, instead of simply verifying the external hash. If the option
    --no-backend-verification is set, no remote files are verified. This option is
    automatically set when then verification is performed directly. ListAndIndexes
    is like True but only dlist and index volumes are handled.
    * values: True, False, ListAndIndexes, IndexesOnly, IndexOnly
    * default value: False

The new part that would suit index file repairs is IndexesOnly, which avoids dblock data fetch. Checking all index files at once would have you ask all (so not sampled) on test command.

Another option (which won’t do as well on fixing index files faster than sampling) is to use this:

  --suppress-warnings (String): Suppress specific warnings
    Suppress warnings and log them as information instead. Use this if you need to
    silence specific warnings. This option accepts a comma separated list of
    warning IDs.

and here the warning ID would be FaultyIndexFiles. They still get fixed, but it runs silently.