Duplicati Robustness & Reliability

I would like to share my concerns regarding Duplicati robustness and reliability:

Backgroud:
I use Duplicati for 18 months (since Mozy sale to Carbonite was announced).
My backup is 90 Gb, has two big PST files, around 50.000 files and 86 versions.
I run 2 backup sets with the exact same configuration (but different repositories) for any failure.
My Duplicati version is 2.0.4.5_beta_2018-11-28

Since the beginning of the year I had two main failures:

  • The first one was: “Detected non-empty blocksets with no associated blocks!”
    That issue was due because I had to go before backup process end. Therefore I slept my laptop to take it with me. When I turned-it on again, it results in ending the backup process unexpectedly.
    I have not been able to retrieve the backup. The repair DB command was useless and the recreate DB one endless.
    I solve this issue copying the working backup’s files from the 2nd repository. Takes 30 hours.

  • The second one is: “Unexpected difference in fileset version X. XX/XX/2019. (Database id: xxx) found xxx entries but expected xxx”
    That error occurs in the same context (had to leave) but that time I stopped the backup and wait until duplicati end the stopping process to sleep my computer.
    I am trying to solve that issue with this post but is the 18th version I am deleting and still getting the same error as here.
    I will let you know If I have been able to “save” that backup.

Let me tell you that I love that project, I love opensource, I have make donations to duplicati, the community has always been there when I need it (even more quicker that Mozy support) and I fully understand that only volunteers works on that project. That being said:

SORRY BUT SOMETHING IS WRONG
It is unthinkable that an event like an unexpected backup interruption or even more a requested backup interruption end with fatal error that let your backup unusable.

3 Likes

My previous post was not constructive. (even if true)
Considering @ts678 comment in that post

My Duplicati version is 2.0.4.5_beta_2018-11-28

Below the backup log when I interrupt the back-up process (The second failure mentioned in the first post)

27 de jun. de 2019 20:06: Result
DeletedFiles: 1
DeletedFolders: 0
ModifiedFiles: 38
ExaminedFiles: 36984
OpenedFiles: 41
AddedFiles: 3
SizeOfModifiedFiles: 29642607215
SizeOfAddedFiles: 137185
SizeOfExaminedFiles: 100659818461
SizeOfOpenedFiles: 29642839736
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
CompactResults:
DeletedFileCount: 0
DownloadedFileCount: 0
UploadedFileCount: 0
DeletedFileSize: 0
DownloadedFileSize: 0
UploadedFileSize: 0
Dryrun: False
MainOperation: Compact
ParsedResult: Success
Version: 2.0.4.5 (2.0.4.5_beta_2018-11-28)
EndTime: 01/01/0001 1:00:00 (-62135596800)
BeginTime: 27/06/2019 19:31:30 (1561656690)
Duration: 00:00:00
Messages: [
2019-06-27 19:17:45 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: La operación Backup ha comenzado,
2019-06-27 19:19:23 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: (),
2019-06-27 19:19:31 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (3,94 KB),
2019-06-27 19:29:33 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),
2019-06-27 19:30:12 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),

]
Warnings:
Errors:
BackendStatistics:
RemoteCalls: 199
BytesUploaded: 1060577631
BytesDownloaded: 2131960446
FilesUploaded: 43
FilesDownloaded: 54
FilesDeleted: 101
FoldersCreated: 0
RetryAttempts: 0
UnknownFileSize: 0
UnknownFileCount: 0
KnownFileCount: 4031
KnownFileSize: 100885933923
LastBackupDate: 26/06/2019 20:45:21 (1561574721)
BackupListCount: 86
TotalQuotaSpace: 0
FreeQuotaSpace: 0
AssignedQuotaSpace: -1
ReportedQuotaError: False
ReportedQuotaWarning: False
ParsedResult: Success
Version: 2.0.4.5 (2.0.4.5_beta_2018-11-28)
Messages: [
2019-06-27 19:17:45 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: La operación Backup ha comenzado,
2019-06-27 19:19:23 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: (),
2019-06-27 19:19:31 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (3,94 KB),
2019-06-27 19:29:33 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),
2019-06-27 19:30:12 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),

]
Warnings:
Errors:
DeleteResults:
DeletedSets: [
Item1: 61
Item2: 27/03/2019 18:41:54 (1553708514)
]
Dryrun: False
MainOperation: Delete
ParsedResult: Success
Version: 2.0.4.5 (2.0.4.5_beta_2018-11-28)
EndTime: 27/06/2019 20:06:06 (1561658766)
BeginTime: 27/06/2019 19:30:23 (1561656623)
Duration: 00:35:43.4163050
Messages: [
2019-06-27 19:17:45 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: La operación Backup ha comenzado,
2019-06-27 19:19:23 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: (),
2019-06-27 19:19:31 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (3,94 KB),
2019-06-27 19:29:33 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),
2019-06-27 19:30:12 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),

]
Warnings:
Errors:
RepairResults: null
TestResults: null
ParsedResult: Success
Version: 2.0.4.5 (2.0.4.5_beta_2018-11-28)
EndTime: 27/06/2019 20:06:06 (1561658766)
BeginTime: 27/06/2019 19:17:45 (1561655865)
Duration: 00:48:21.0699620
Messages: [
2019-06-27 19:17:45 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: La operación Backup ha comenzado,
2019-06-27 19:19:23 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: (),
2019-06-27 19:19:31 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (3,94 KB),
2019-06-27 19:29:33 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),
2019-06-27 19:30:12 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bcb6317c5865140c59a3d9ad9958ab7a2.dblock.zip.aes (49,91 MB),

]
Warnings:
Errors:

Let me know if I can provide any additional details.

Making the follow-up on:

I have not been able to recover my backup.
I will apply the same solution as for the first failure.