Hi I have multiple backups backing up to Backblaze B2. These backups have been stable for ~2 years. All of a sudden this weekend all backups are throwing verification errors.
I’m running a dockerized container version: 2.0.5.1_beta_2020-01-18
2021-03-28 00:14:23 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-20200728T040000Z.dlist.zip.aes
2021-03-28 00:16:17 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-i90cdc4f25eed42e58d09563adf9f0623.dindex.zip.aes
2021-03-28 00:18:08 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b88feb6e2071d4456a1ddce690b118d14.dblock.zip.aes
The JSON output of the error is:
{
"DeletedFiles": 0,
"DeletedFolders": 0,
"ModifiedFiles": 0,
"ExaminedFiles": 56400,
"OpenedFiles": 0,
"AddedFiles": 0,
"SizeOfModifiedFiles": 0,
"SizeOfAddedFiles": 0,
"SizeOfExaminedFiles": 207558947669,
"SizeOfOpenedFiles": 0,
"NotProcessedFiles": 0,
"AddedFolders": 0,
"TooLargeFiles": 0,
"FilesWithError": 0,
"ModifiedFolders": 0,
"ModifiedSymlinks": 0,
"AddedSymlinks": 0,
"DeletedSymlinks": 0,
"PartialBackup": false,
"Dryrun": false,
"MainOperation": "Backup",
"CompactResults": null,
"VacuumResults": null,
"DeleteResults": {
"DeletedSetsActualLength": 0,
"DeletedSets": [],
"Dryrun": false,
"MainOperation": "Delete",
"CompactResults": null,
"ParsedResult": "Success",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "2021-03-28T04:12:15.853472Z",
"BeginTime": "2021-03-28T04:09:24.519867Z",
"Duration": "00:02:51.3336050",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 17,
"BytesUploaded": 0,
"BytesDownloaded": 0,
"FilesUploaded": 0,
"FilesDownloaded": 0,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 12,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 6980,
"KnownFileSize": 181938235700,
"LastBackupDate": "2021-03-15T00:00:00-04:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2021-03-28T04:00:00.032973Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"RepairResults": null,
"TestResults": {
"MainOperation": "Test",
"VerificationsActualLength": 3,
"Verifications": [
{
"Key": "duplicati-20200728T040000Z.dlist.zip.aes",
"Value": [
{
"Key": "Error",
"Value": "Error: ConnectFailure (The requested address is not valid in this context)"
}
]
},
{
"Key": "duplicati-i90cdc4f25eed42e58d09563adf9f0623.dindex.zip.aes",
"Value": [
{
"Key": "Error",
"Value": "Error: ConnectFailure (The requested address is not valid in this context)"
}
]
},
{
"Key": "duplicati-b88feb6e2071d4456a1ddce690b118d14.dblock.zip.aes",
"Value": [
{
"Key": "Error",
"Value": "Error: ConnectFailure (The requested address is not valid in this context)"
}
]
}
],
"ParsedResult": "Success",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "2021-03-28T04:18:08.776795Z",
"BeginTime": "2021-03-28T04:12:34.270568Z",
"Duration": "00:05:34.5062270",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 17,
"BytesUploaded": 0,
"BytesDownloaded": 0,
"FilesUploaded": 0,
"FilesDownloaded": 0,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 12,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 6980,
"KnownFileSize": 181938235700,
"LastBackupDate": "2021-03-15T00:00:00-04:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2021-03-28T04:00:00.032973Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"ParsedResult": "Error",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "2021-03-28T04:18:08.777094Z",
"BeginTime": "2021-03-28T04:00:00.032968Z",
"Duration": "00:18:08.7441260",
"MessagesActualLength": 41,
"WarningsActualLength": 0,
"ErrorsActualLength": 3,
"Messages": [
"2021-03-28 00:00:00 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started",
"2021-03-28 00:07:46 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2021-03-28 00:08:03 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (6.82 KB)",
"2021-03-28 00:09:24 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-StartCheck]: Start checking if backups can be removed",
"2021-03-28 00:09:24 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-FramesAndIntervals]: Time frames and intervals pairs: 7.00:00:00 / 1.00:00:00, 28.00:00:00 / 7.00:00:00, 1095.00:00:00 / 31.00:00:00",
"2021-03-28 00:09:24 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupList]: Backups to consider: 3/8/2021 11:00:00 PM, 2/6/2021 11:00:00 PM, 9/27/2020 12:00:00 AM, 7/28/2020 12:00:00 AM, 6/18/2020 5:09:19 PM",
"2021-03-28 00:09:24 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsToDelete]: Backups outside of all time frames and thus getting deleted: ",
"2021-03-28 00:09:24 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-AllBackupsToDelete]: All backups to delete: ",
"2021-03-28 00:12:15 -04 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteResults]: No remote filesets were deleted",
"2021-03-28 00:12:15 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2021-03-28 00:12:32 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (6.82 KB)",
"2021-03-28 00:12:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:12:48 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:12:58 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:13:11 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:13:21 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:13:36 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:13:46 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:13:59 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)",
"2021-03-28 00:14:09 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200728T040000Z.dlist.zip.aes (5.03 MB)"
],
"Warnings": [],
"Errors": [
"2021-03-28 00:14:23 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-20200728T040000Z.dlist.zip.aes",
"2021-03-28 00:16:17 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-i90cdc4f25eed42e58d09563adf9f0623.dindex.zip.aes",
"2021-03-28 00:18:08 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b88feb6e2071d4456a1ddce690b118d14.dblock.zip.aes"
],
"BackendStatistics": {
"RemoteCalls": 17,
"BytesUploaded": 0,
"BytesDownloaded": 0,
"FilesUploaded": 0,
"FilesDownloaded": 0,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 12,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 6980,
"KnownFileSize": 181938235700,
"LastBackupDate": "2021-03-15T00:00:00-04:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.5.1 (2.0.5.1_beta_2020-01-18)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2021-03-28T04:00:00.032973Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
}
I’ve tried:
- Database Repair, then Recreate
- A new test backup to a new B2 bucket
- Install an older docker container ver for Duplicati
None of these have worked.
I then tried the option: backup-test-samples = 0 and this stopped the errors, but I’m worried that this may not be a safe solution.
I know Backblaze recently announced Server sided encryption but I don’t know how that would suddenly affect my stable setup.
Any thought or suggestions would be greatly appreciated.
Tx