Missing remote files

[jimkd1yv]

I got some missing remote file errors. I followed instructions in other threads to purge-broken-files; then Recreate (delete & repair) the database. However that step throws 4 missing remote files.
(Note that this is a different machine than my problem reported last week)

Complete log:
{
“MainOperation”: “Repair”,
“RecreateDatabaseResults”: {
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T15:54:33.228334Z”,
“BeginTime”: “2020-02-25T15:18:58.197757Z”,
“Duration”: “00:35:35.0305770”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null,
“BackendStatistics”: {
“RemoteCalls”: 436,
“BytesUploaded”: 0,
“BytesDownloaded”: 347975799,
“FilesUploaded”: 0,
“FilesDownloaded”: 435,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 0,
“KnownFileSize”: 0,
“LastBackupDate”: “0001-01-01T00:00:00”,
“BackupListCount”: 0,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: 0,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T15:18:58.042646Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
},
“ParsedResult”: “Error”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T15:54:33.881191Z”,
“BeginTime”: “2020-02-25T15:18:58.042637Z”,
“Duration”: “00:35:35.8385540”,
“MessagesActualLength”: 878,
“WarningsActualLength”: 3,
“ErrorsActualLength”: 4,
“Messages”: [
“2020-02-25 10:18:58 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started”,
“2020-02-25 10:18:58 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2020-02-25 10:18:59 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (715 bytes)”,
“2020-02-25 10:19:02 -05 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 149 filelists”,
“2020-02-25 10:19:02 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190602T012317Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190602T012317Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190603T025619Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190603T025619Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190725T224500Z.dlist.zip.aes (59.90 KB)”,
“2020-02-25 10:19:04 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190725T224500Z.dlist.zip.aes (59.90 KB)”,
“2020-02-25 10:19:04 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190805T224500Z.dlist.zip.aes (60.06 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190805T224500Z.dlist.zip.aes (60.06 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190806T224500Z.dlist.zip.aes (60.73 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190806T224500Z.dlist.zip.aes (60.73 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190821T031613Z.dlist.zip.aes (68.81 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190821T031613Z.dlist.zip.aes (68.81 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190829T032609Z.dlist.zip.aes (83.50 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190829T032609Z.dlist.zip.aes (83.50 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190830T020725Z.dlist.zip.aes (83.62 KB)”,
“2020-02-25 10:19:07 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190830T020725Z.dlist.zip.aes (83.62 KB)”
],
“Warnings”: [
“2020-02-25 10:18:58 -05 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option --auth-password is not supported and will be ignored”,
“2020-02-25 10:52:35 -05 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 4 missing volumes; attempting to replace blocks from existing volumes”,
“2020-02-25 10:53:42 -05 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 4 missing volumes; attempting to replace blocks from existing volumes”
],
“Errors”: [
“2020-02-25 10:49:13 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b55de5cb436404990b572152b59fe7b0d.dblock.zip.aes by duplicati-i46065ac0f2984b3b9bbc9229069ff30f.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:50:08 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-bb5f67341722a41538da71cc11c1af640.dblock.zip.aes by duplicati-i41ba232ed7a54d2e8a67276083e5c6aa.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:50:14 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b1b8d6924cf88412ea95c7b33366071c9.dblock.zip.aes by duplicati-idf5c3d28ba7e4dbc9a0cde1eb9056c61.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:51:09 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b8cd946897ef64b9c92a8a1692f89e69b.dblock.zip.aes by duplicati-iffa5f24683c6425fb85eb8b216dc96b8.dindex.zip.aes, but not found in list, registering a missing remote file”
],
“BackendStatistics”: {
“RemoteCalls”: 436,
“BytesUploaded”: 0,
“BytesDownloaded”: 347975799,
“FilesUploaded”: 0,
“FilesDownloaded”: 435,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 0,
“KnownFileSize”: 0,
“LastBackupDate”: “0001-01-01T00:00:00”,
“BackupListCount”: 0,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: 0,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T15:18:58.042646Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
}

[ts678]

Have you been on Canary for awhile? If so, thanks for taking some extra risk to shake bugs out early.

dblock put retry corrupts dindex, using old dblock name for index – canary regression #3932
is a possible explanation for this. The bug was present in Canary from v2.0.4.16-2.0.4.16_canary_2019-03-28 and fixed in v2.0.4.31-2.0.4.31_canary_2019-10-19

Fixed a retry error where uploaded dindex -files would reference non-existing dblock files, thanks @warwickmm

however you probably wouldn’t notice an issue until Recreate time because database is used instead.

Is there anything seemingly working wrong, or just some errors? If you want to remove the errors so a future Recreate is clean, you might be able to do so by hiding the 4 complaining dindex files, e.g. by a move elsewhere or prefix with something different, e.g. hidden-, and see if Repair can recreate them.

If it does, and you’re feeling experimental, try Recreate again and see if it’s better. The new dindex will come from current database records, and I think after downloading 435 files and passing a consistency check at the end, Recreate found the missing block even though the dindex file pointed the wrong way.

You can sanity-check your destination files by name. If you have the same number of files with dblock in their name as you have dindex files, you probably don’t have any actually missing dblock files there, simply ones that the dindex named when it should have named the successor dblock file on the retry…

[jimkd1yv]

I ran the repair, then 1 successful backup (with just 1 common warning). However, the next day’s scheduled backup threw missing entry errors. I have not been good about keeping old logs and errors, since repairing the DB deketes the logs. However, I will begin saving them before performing future repairs.

If I try to run the backup manually now, it throws the same error, but does not run, nor does it produce any log.

Here are the logs that I currently have:
Repair on 25 Feb:
{
“MainOperation”: “Repair”,
“RecreateDatabaseResults”: {
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T15:54:33.228334Z”,
“BeginTime”: “2020-02-25T15:18:58.197757Z”,
“Duration”: “00:35:35.0305770”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null,
“BackendStatistics”: {
“RemoteCalls”: 436,
“BytesUploaded”: 0,
“BytesDownloaded”: 347975799,
“FilesUploaded”: 0,
“FilesDownloaded”: 435,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 0,
“KnownFileSize”: 0,
“LastBackupDate”: “0001-01-01T00:00:00”,
“BackupListCount”: 0,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: 0,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T15:18:58.042646Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
},
“ParsedResult”: “Error”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T15:54:33.881191Z”,
“BeginTime”: “2020-02-25T15:18:58.042637Z”,
“Duration”: “00:35:35.8385540”,
“MessagesActualLength”: 878,
“WarningsActualLength”: 3,
“ErrorsActualLength”: 4,
“Messages”: [
“2020-02-25 10:18:58 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started”,
“2020-02-25 10:18:58 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2020-02-25 10:18:59 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (715 bytes)”,
“2020-02-25 10:19:02 -05 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 149 filelists”,
“2020-02-25 10:19:02 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190602T012317Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190602T012317Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190603T025619Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190603T025619Z.dlist.zip.aes (59.79 KB)”,
“2020-02-25 10:19:03 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190725T224500Z.dlist.zip.aes (59.90 KB)”,
“2020-02-25 10:19:04 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190725T224500Z.dlist.zip.aes (59.90 KB)”,
“2020-02-25 10:19:04 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190805T224500Z.dlist.zip.aes (60.06 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190805T224500Z.dlist.zip.aes (60.06 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190806T224500Z.dlist.zip.aes (60.73 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190806T224500Z.dlist.zip.aes (60.73 KB)”,
“2020-02-25 10:19:05 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190821T031613Z.dlist.zip.aes (68.81 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190821T031613Z.dlist.zip.aes (68.81 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190829T032609Z.dlist.zip.aes (83.50 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190829T032609Z.dlist.zip.aes (83.50 KB)”,
“2020-02-25 10:19:06 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20190830T020725Z.dlist.zip.aes (83.62 KB)”,
“2020-02-25 10:19:07 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20190830T020725Z.dlist.zip.aes (83.62 KB)”
],
“Warnings”: [
“2020-02-25 10:18:58 -05 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option --auth-password is not supported and will be ignored”,
“2020-02-25 10:52:35 -05 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 4 missing volumes; attempting to replace blocks from existing volumes”,
“2020-02-25 10:53:42 -05 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 4 missing volumes; attempting to replace blocks from existing volumes”
],
“Errors”: [
“2020-02-25 10:49:13 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b55de5cb436404990b572152b59fe7b0d.dblock.zip.aes by duplicati-i46065ac0f2984b3b9bbc9229069ff30f.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:50:08 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-bb5f67341722a41538da71cc11c1af640.dblock.zip.aes by duplicati-i41ba232ed7a54d2e8a67276083e5c6aa.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:50:14 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b1b8d6924cf88412ea95c7b33366071c9.dblock.zip.aes by duplicati-idf5c3d28ba7e4dbc9a0cde1eb9056c61.dindex.zip.aes, but not found in list, registering a missing remote file”,
“2020-02-25 10:51:09 -05 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-b8cd946897ef64b9c92a8a1692f89e69b.dblock.zip.aes by duplicati-iffa5f24683c6425fb85eb8b216dc96b8.dindex.zip.aes, but not found in list, registering a missing remote file”
],
“BackendStatistics”: {
“RemoteCalls”: 436,
“BytesUploaded”: 0,
“BytesDownloaded”: 347975799,
“FilesUploaded”: 0,
“FilesDownloaded”: 435,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 0,
“KnownFileSize”: 0,
“LastBackupDate”: “0001-01-01T00:00:00”,
“BackupListCount”: 0,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: 0,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Repair”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T15:18:58.042646Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
}

=~=~=~=~=~=
Manual back run 25 Feb. after repair
{
“DeletedFiles”: 6,
“DeletedFolders”: 14,
“ModifiedFiles”: 257,
“ExaminedFiles”: 40553,
“OpenedFiles”: 34540,
“AddedFiles”: 110,
“SizeOfModifiedFiles”: 1165195291,
“SizeOfAddedFiles”: 67852624,
“SizeOfExaminedFiles”: 12183209572,
“SizeOfOpenedFiles”: 9794576760,
“NotProcessedFiles”: 0,
“AddedFolders”: 52,
“TooLargeFiles”: 0,
“FilesWithError”: 0,
“ModifiedFolders”: 0,
“ModifiedSymlinks”: 0,
“AddedSymlinks”: 0,
“DeletedSymlinks”: 0,
“PartialBackup”: false,
“Dryrun”: false,
“MainOperation”: “Backup”,
“CompactResults”: null,
“VacuumResults”: null,
“DeleteResults”: null,
“RepairResults”: null,
“TestResults”: {
“MainOperation”: “Test”,
“VerificationsActualLength”: 3,
“Verifications”: [
{
“Key”: “duplicati-20191125T040057Z.dlist.zip.aes”,
“Value”:
},
{
“Key”: “duplicati-i815ba8b1dfb342dfa69b07f4789ffc9a.dindex.zip.aes”,
“Value”:
},
{
“Key”: “duplicati-b8f32014394084381947a021638828568.dblock.zip.aes”,
“Value”:
}
],
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T20:00:25.37042Z”,
“BeginTime”: “2020-02-25T19:59:52.187013Z”,
“Duration”: “00:00:33.1834070”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null,
“BackendStatistics”: {
“RemoteCalls”: 8,
“BytesUploaded”: 53442103,
“BytesDownloaded”: 53795975,
“FilesUploaded”: 3,
“FilesDownloaded”: 3,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 718,
“KnownFileSize”: 12380357558,
“LastBackupDate”: “2020-02-25T14:42:58-05:00”,
“BackupListCount”: 150,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: -1,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Backup”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T19:42:58.679899Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
},
“ParsedResult”: “Warning”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “2020-02-25T20:00:25.379702Z”,
“BeginTime”: “2020-02-25T19:42:58.679891Z”,
“Duration”: “00:17:26.6998110”,
“MessagesActualLength”: 21,
“WarningsActualLength”: 1,
“ErrorsActualLength”: 0,
“Messages”: [
“2020-02-25 14:42:58 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started”,
“2020-02-25 14:43:48 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2020-02-25 14:43:51 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (715 bytes)”,
“2020-02-25 14:43:51 -05 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-RemoteUnwantedMissingFile]: removing file listed as Temporary: duplicati-b55de5cb436404990b572152b59fe7b0d.dblock.zip.aes”,
“2020-02-25 14:43:51 -05 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-RemoteUnwantedMissingFile]: removing file listed as Temporary: duplicati-bb5f67341722a41538da71cc11c1af640.dblock.zip.aes”,
“2020-02-25 14:43:51 -05 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-RemoteUnwantedMissingFile]: removing file listed as Temporary: duplicati-b1b8d6924cf88412ea95c7b33366071c9.dblock.zip.aes”,
“2020-02-25 14:43:51 -05 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-RemoteUnwantedMissingFile]: removing file listed as Temporary: duplicati-b8cd946897ef64b9c92a8a1692f89e69b.dblock.zip.aes”,
“2020-02-25 14:49:41 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b34dd4c26d349462aa77832d095f1f603.dblock.zip.aes (48.17 MB)”,
“2020-02-25 14:49:45 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-20200225T194258Z.dlist.zip.aes (2.36 MB)”,
“2020-02-25 14:51:13 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-20200225T194258Z.dlist.zip.aes (2.36 MB)”,
“2020-02-25 14:59:41 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b34dd4c26d349462aa77832d095f1f603.dblock.zip.aes (48.17 MB)”,
“2020-02-25 14:59:42 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i12208610e6eb4d4593adf0e54e3444f9.dindex.zip.aes (450.61 KB)”,
“2020-02-25 14:59:50 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i12208610e6eb4d4593adf0e54e3444f9.dindex.zip.aes (450.61 KB)”,
“2020-02-25 14:59:50 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2020-02-25 14:59:52 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (718 bytes)”,
“2020-02-25 14:59:52 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20191125T040057Z.dlist.zip.aes (1.32 MB)”,
“2020-02-25 14:59:53 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20191125T040057Z.dlist.zip.aes (1.32 MB)”,
“2020-02-25 14:59:53 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-i815ba8b1dfb342dfa69b07f4789ffc9a.dindex.zip.aes (38.45 KB)”,
“2020-02-25 14:59:54 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-i815ba8b1dfb342dfa69b07f4789ffc9a.dindex.zip.aes (38.45 KB)”,
“2020-02-25 14:59:54 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b8f32014394084381947a021638828568.dblock.zip.aes (49.94 MB)”
],
“Warnings”: [
“2020-02-25 14:42:58 -05 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option --auth-password is not supported and will be ignored”
],
“Errors”: ,
“BackendStatistics”: {
“RemoteCalls”: 8,
“BytesUploaded”: 53442103,
“BytesDownloaded”: 53795975,
“FilesUploaded”: 3,
“FilesDownloaded”: 3,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 0,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 718,
“KnownFileSize”: 12380357558,
“LastBackupDate”: “2020-02-25T14:42:58-05:00”,
“BackupListCount”: 150,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: -1,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Backup”,
“ParsedResult”: “Success”,
“Version”: “2.0.5.101 (2.0.5.101_canary_2020-01-23)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2020-02-25T19:42:58.679899Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
}

Today’s scheduled backup – No log generated, just an Error Message:

[ts678]

@jimkd1yv

I ran the repair, then 1 successful backup (with just 1 common warning). However, the next day’s scheduled backup threw missing entry errors.

It’s odd that there was 1 successful one first. At one time there was a bug in Compacting files at the backend that can happen after backup, but that’s fixed in 2.0.5.1, and I also see no compact logged.

What used to happen is the check before backup was fine, the one after was fine, compact ran and caused damage, then the next backup would complain at its start, but that doesn’t seem to fit here…

Maybe one of the usual attempts at a fix is worth trying, though sometimes the problem just moves…

You could open Commandline for job, change Command to Delete, change Commandline arguments box to --version=130 (assuming that’s still the error), then Run "delete" command now at bottom.

[jimkd1yv]

Ran the delete, got this result:
Finished!

The supplied option --auth-password is not supported and will be ignored
Listing remote folder …
Deleting file duplicati-20191007T224500Z.dlist.zip.aes (1.21 MB) …
Deleting file duplicati-bf910775c9c554f1daeabcf3c2f1fd838.dblock.zip.aes (49.97 MB) …
Deleting file duplicati-i2c9b4612bb63462f9b1b4cb4ff03412b.dindex.zip.aes (829 bytes) …
Deleting file duplicati-b7abb8fc268a7443ba70a919bde82aeb8.dblock.zip.aes (26.64 MB) …
Deleting file duplicati-ic2f33a6ccaff4a6b8ba810aee48c3613.dindex.zip.aes (829 bytes) …
Deleting file duplicati-bc009537fcfad4c719b90dd92fa10fe9d.dblock.zip.aes (49.91 MB) …
Deleting file duplicati-i48faa8f7f4f447e7b24aa824f228f792.dindex.zip.aes (829 bytes) …
These filesets were deleted:
130: 10/7/2019 6:45:00 PM
Return code: 0

xxxxxxxxxxxxxxx
Then tried to run a backup, got this error. Note data file number decremented and date has incremented. Shall I go ahead and delete this one as well?

[ts678]

Unfortunately it errors on the first one it finds, instead of continuing to see how many others are bad. Changing date implies it’s actually a different version, and you didn’t somehow name the wrong one.

There’s no easy way to tell how far this is going to go. Blocks go forward as needed (avoids upload). Unfortunately, lack of blocks (if that is what this is) can also carry forward. Might as well try to delete.

Studying the database with SQL queries (I can provide sample) can test other versions, but it’s hard.

Surest way to avoid problems might be to delete Database and remote files for backup from scratch, however sometimes old history is valuable. If space is available, damaged one could also be saved.
The RESTORE command is more forgiving, but possibly even normal Restores will run pretty well…

@jimkd1yv

I got some missing remote file errors.

Any idea what might have happened to encourage them, e.g. system crash or other hard shutdown? Previous fix target was the compact operation. Work is still in progress on handling hard stops better.

[jimkd1yv]

OK, I just want to be sure I understand what you are recommending:

1. From local installation, select current backup | database | Delete
2. From remote repository (dropbox in my case), sign on and delete all files for this backup
3. From local installation, run a brand-new baseline backup.

Kindly confirm
Thanks

[ts678]

That’s the basic path, and the easiest one (actually a folder delete and recreate might also be simple).

Everything below is optional. If you have space, the old remote backup could be left, and the new one made nearby. The old one might be useful if you someday want an old version.The tool I meant to cite:

Duplicati.CommandLine.RecoveryTool.exe

This tool can be used in very specific situations, where you have to restore data from a corrupted backup. The procedure for recovering from this scenario is covered in Disaster Recovery.

If you have space locally, backups can be moved as you wish. Even better is if the backup’s history is less important. Definitely be careful of Canary on important ones. Even Beta has risk, but getting less.

For the database, a manual rename of the path shown on the Database tab could also be done if you think you might want to fight this further (and also keep the old remote). Easiest path is Delete button.

The --auth-password error is usually a leftover either from a previous destination that used one, maybe even temporarily at initial creation where a password saver got too eager and put it on the destination.

on the Destination screen will let you edit it off the end of the URL if it’s at the end and doesn’t belong.