Tardigrade.io - Missing dblocks

Support
1

I’m running a backup task to tardigrade and I’m getting now an error → see below. None of the operations like repair are working. Any ideas?

Background:
-Docker Installation on Synology NAS.
-2.0.5.111_canary_2020-09-26
-MonoVersion : 5.20.1.34

Found 1 files that are missing from the remote storage, please run repair

Log:

Duplicati.Library.Interface.UserInformationException: Found 1 files that are missing from the remote storage, please run repair
  at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.Collections.Generic.IEnumerable`1[T] protectedFiles) [0x001fd] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x0011d] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x01048] in <69351e729b034105bffb62d55f82ef1c>:0 
  at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <9a758ff4db6c48d6b3d4d0e5c2adf6d1>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x00009] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass14_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x0004b] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller.Backup (System.String[] inputsources, Duplicati.Library.Utility.IFilter filter) [0x00074] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00349] in <d348e18e6cb747ab98d1065177fd004f>:0
2

First step is probably to find out which file is missing. Watch About → Show log → Live → Warning fail.
<backup> → Show log → Remote might be able to awkwardly see whether Duplicati tried to do upload.

Is this an initial backup or one with long history on it that failed? Anything unusual in job logs around then?
Because your stack is in PreBackupVerify, whatever went wrong probably went wrong on prior backup…

If problem is easily reproducible from scratch, setting log-file=<path> at log-file-log-level=retry can debug.

EDIT:

If it’s an older backup, I suppose you could look for something like DB Browser for SQLite (sqlitebrowser).
Your error looks like Remotevolume table says it has an uploaded file that it isn’t seeing in Tardigrade now.
RemoteOperation table can maybe show when file was uploaded, but log file from scratch may be easier.

What happens?

3

When I try to run repair I see the following errors:

Nov 20, 2020 11:12 AM: Failed while executing "Repair" with id: 5
Duplicati.Library.Interface.UserInformationException: The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. **The following files are missing: duplicati-baf382f70556840a3935ab132363a79e1.dblock.zip.aes**
  at Duplicati.Library.Main.Operation.RepairHandler.RunRepairRemote () [0x008d8] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Operation.RepairHandler.Run (Duplicati.Library.Utility.IFilter filter) [0x0016d] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass18_0.<Repair>b__0 (Duplicati.Library.Main.RepairResults result) [0x0001c] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Library.Main.Controller.Repair (Duplicati.Library.Utility.IFilter filter) [0x0001a] in <69351e729b034105bffb62d55f82ef1c>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x003ad] in <d348e18e6cb747ab98d1065177fd004f>:0

I see that a zip file is missing, but what do I do with that information? Since everything is encrypted I have no idea what actual files these are.

Here is my amendet backup script:

{
“CreatedByVersion”: “2.0.5.111”,
“Schedule”: {
“ID”: 1,
“Tags”: [
“ID=5”
],
“Time”: “2020-11-21T14:00:00Z”,
“Repeat”: “1D”,
“LastRun”: “2020-11-19T14:00:00Z”,
“Rule”: “AllowedWeekDays=Tuesday,Thursday,Saturday”,
“AllowedDays”: [
“tue”,
“thu”,
“sat”
]
},
“Backup”: {
“ID”: “5”,
“Name”: “xxxxxx”,
“Description”: “xxxxxx”,
“Tags”: ,
“TargetURL”: “tardigrade://tardigrade.io/config?tardigrade-auth-method=API%20key&tardigrade-satellite=europe-west-1.tardigrade.io%3A7777&tardigrade-api-key=xxxxxx-bucket=xxxx”,
“DBPath”: “/data/Duplicati/HSTUOBFSGY.sqlite”,
“Sources”: [
“/xxxx/xxxx/”
],
“Settings”: [
{
“Filter”: “”,
“Name”: “encryption-module”,
“Value”: “aes”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “compression-module”,
“Value”: “zip”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “dblock-size”,
“Value”: “64MB”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “retention-policy”,
“Value”: “1W:1D,4W:1W,12M:1M”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–asynchronous-concurrent-upload-limit”,
“Value”: “1”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–backup-test-samples”,
“Value”: “0”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–list-verify-uploads”,
“Value”: “true”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–no-auto-compact”,
“Value”: “true”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–threshold”,
“Value”: “5”,
“Argument”: null
},
{
“Filter”: “”,
“Name”: “–exclude-files-attributes”,
“Value”: “hidden,system,temporary”,
“Argument”: null
}
],
“Filters”: [
{
“Order”: 0,
“Include”: false,
“Expression”: “xxxx/”
},
{
“Order”: 1,
“Include”: false,
“Expression”: “/xxxx/”
},
{
“Order”: 2,
“Include”: false,
“Expression”: “/xxxx/”
}
],
“Metadata”: {
“LastErrorDate”: “20201120T101306Z”,
“LastErrorMessage”: “The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-baf382f70556840a3935ab132363a79e1.dblock.zip.aes”,
“LastBackupDate”: “20201116T175334Z”,
“BackupListCount”: “3”,
“TotalQuotaSpace”: “0”,
“FreeQuotaSpace”: “0”,
“AssignedQuotaSpace”: “-1”,
“TargetFilesSize”: “188665416751”,
“TargetFilesCount”: “5627”,
“TargetSizeString”: “175.71 GB”,
“SourceFilesSize”: “230572427791”,
“SourceFilesCount”: “57915”,
“SourceSizeString”: “214.74 GB”,
“LastBackupStarted”: “20201117T140000Z”,
“LastBackupFinished”: “20201117T143937Z”,
“LastBackupDuration”: “00:39:37.1470120”
},
“IsTemporary”: false
},
“DisplayNames”: {
“/xxxx/”: “xxxx”
}
}

4

The AFFECTED command in GUI Commandline can help. See example of its usage.

Returns a report explaining what backup sets and files are affected by a remote file.

It would be nice to figure out how this happened before Tardigrade progresses from Canary into Beta, however DB or log study might be needed, and if it’s in Tardigrade, help of developer may be required.

5

Well I have to close the thread. I solved it by starting fresh. And since then it is running smooth with several backups to different buckets. :slight_smile: