Error while running Synology backup

Support
1

I keep getting this error. How do I fix this?

2024-02-08 11:23:22 -05 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b8d4e45575ba545c1b0c87091cc6fccb7.dblock.zip.aes

2 
        {

“DeletedFiles”: 0,
“DeletedFolders”: 0,
“ModifiedFiles”: 0,
“ExaminedFiles”: 5049,
“OpenedFiles”: 5048,
“AddedFiles”: 5048,
“SizeOfModifiedFiles”: 0,
“SizeOfAddedFiles”: 22045493038,
“SizeOfExaminedFiles”: 22045493203,
“SizeOfOpenedFiles”: 22045493038,
“NotProcessedFiles”: 0,
“AddedFolders”: 3517,
“TooLargeFiles”: 0,
“FilesWithError”: 0,
“ModifiedFolders”: 0,
“ModifiedSymlinks”: 0,
“AddedSymlinks”: 0,
“DeletedSymlinks”: 0,
“PartialBackup”: false,
“Dryrun”: false,
“MainOperation”: “Backup”,
“CompactResults”: null,
“VacuumResults”: null,
“DeleteResults”: {
“DeletedSetsActualLength”: 0,
“DeletedSets”: ,
“Dryrun”: false,
“MainOperation”: “Delete”,
“CompactResults”: null,
“ParsedResult”: “Success”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “2024-02-08T16:21:22.4156946Z”,
“BeginTime”: “2024-02-08T16:21:21.9252997Z”,
“Duration”: “00:00:00.4903949”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null,
“BackendStatistics”: {
“RemoteCalls”: 730,
“BytesUploaded”: 18852937165,
“BytesDownloaded”: 262736795,
“FilesUploaded”: 721,
“FilesDownloaded”: 7,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 4,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 721,
“KnownFileSize”: 18852937165,
“LastBackupDate”: “2024-02-08T09:42:54-05:00”,
“BackupListCount”: 1,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: -1,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Backup”,
“ParsedResult”: “Success”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2024-02-08T14:42:53.7446268Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
},
“RepairResults”: null,
“TestResults”: {
“MainOperation”: “Test”,
“VerificationsActualLength”: 3,
“Verifications”: [
{
“Key”: “duplicati-20240208T144254Z.dlist.zip.aes”,
“Value”:
},
{
“Key”: “duplicati-if4210fc59cf24fb6a76cece919eedf3f.dindex.zip.aes”,
“Value”:
},
{
“Key”: “duplicati-b8d4e45575ba545c1b0c87091cc6fccb7.dblock.zip.aes”,
“Value”: [
{
“Key”: “Error”,
“Value”: “Hash mismatch on file "C:\Users\druss\AppData\Local\Temp\dup-6ea2c6cb-0ec9-4dde-94f9-104d23dab8df", recorded hash: NBCmIvw/pLg28xzlmW2wnuNGzVBnHxZrQ6vBXN4s3LM=, actual hash BtcpcQLCOjMf6SwUBRqdrcxfqY/KZxfcIgv+VnVSXo8=”
}
]
}
],
“ParsedResult”: “Success”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “2024-02-08T16:23:22.6920289Z”,
“BeginTime”: “2024-02-08T16:21:28.4665669Z”,
“Duration”: “00:01:54.2254620”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null,
“BackendStatistics”: {
“RemoteCalls”: 730,
“BytesUploaded”: 18852937165,
“BytesDownloaded”: 262736795,
“FilesUploaded”: 721,
“FilesDownloaded”: 7,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 4,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 721,
“KnownFileSize”: 18852937165,
“LastBackupDate”: “2024-02-08T09:42:54-05:00”,
“BackupListCount”: 1,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: -1,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Backup”,
“ParsedResult”: “Success”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2024-02-08T14:42:53.7446268Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
},
“ParsedResult”: “Error”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “2024-02-08T16:23:22.7369445Z”,
“BeginTime”: “2024-02-08T14:42:53.7446268Z”,
“Duration”: “01:40:28.9923177”,
“MessagesActualLength”: 1467,
“WarningsActualLength”: 1,
“ErrorsActualLength”: 1,
“Messages”: [
“2024-02-08 09:42:53 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started”,
“2024-02-08 09:42:54 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2024-02-08 09:42:54 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: ()”,
“2024-02-08 09:44:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b88e7993108944a119368187dcc489d9c.dblock.zip.aes (49.90 MB)”,
“2024-02-08 09:44:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bfa2b772e723d444cbc6a9826e501171b.dblock.zip.aes (49.97 MB)”,
“2024-02-08 09:44:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bd92d5f1d35ac4f7abd4a5deca367862c.dblock.zip.aes (49.95 MB)”,
“2024-02-08 09:44:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bd4229e4603304baab3f412620609cddc.dblock.zip.aes (49.94 MB)”,
“2024-02-08 09:44:30 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bfa2b772e723d444cbc6a9826e501171b.dblock.zip.aes (49.97 MB)”,
“2024-02-08 09:44:30 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i078f6289ba90457b9ec9d95f1706fbb3.dindex.zip.aes (19.15 KB)”,
“2024-02-08 09:44:30 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b88e7993108944a119368187dcc489d9c.dblock.zip.aes (49.90 MB)”,
“2024-02-08 09:44:30 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i3eed209bc8064de5828987930927da47.dindex.zip.aes (27.33 KB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i078f6289ba90457b9ec9d95f1706fbb3.dindex.zip.aes (19.15 KB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bd4229e4603304baab3f412620609cddc.dblock.zip.aes (49.94 MB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-ba1e79e24730249c981e637993f558e58.dblock.zip.aes (49.99 MB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i3eed209bc8064de5828987930927da47.dindex.zip.aes (27.33 KB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b0986a3f8648a48b29f00a5e810ff7376.dblock.zip.aes (49.98 MB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i0740ede59fd34be3a39155dbe2218d92.dindex.zip.aes (18.89 KB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i0740ede59fd34be3a39155dbe2218d92.dindex.zip.aes (18.89 KB)”,
“2024-02-08 09:44:31 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bf4311dd0b68f40ddb6eb49a9815814a6.dblock.zip.aes (49.94 MB)”,
“2024-02-08 09:44:32 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-bd92d5f1d35ac4f7abd4a5deca367862c.dblock.zip.aes (49.95 MB)”
],
“Warnings”: [
“2024-02-08 10:24:37 -05 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: E:\empower\Root\DMD Training Manual\~$Lead Scoring Template_CBS_12.15.23.xlsx”
],
“Errors”: [
“2024-02-08 11:23:22 -05 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b8d4e45575ba545c1b0c87091cc6fccb7.dblock.zip.aes”
],
“BackendStatistics”: {
“RemoteCalls”: 730,
“BytesUploaded”: 18852937165,
“BytesDownloaded”: 262736795,
“FilesUploaded”: 721,
“FilesDownloaded”: 7,
“FilesDeleted”: 0,
“FoldersCreated”: 0,
“RetryAttempts”: 4,
“UnknownFileSize”: 0,
“UnknownFileCount”: 0,
“KnownFileCount”: 721,
“KnownFileSize”: 18852937165,
“LastBackupDate”: “2024-02-08T09:42:54-05:00”,
“BackupListCount”: 1,
“TotalQuotaSpace”: 0,
“FreeQuotaSpace”: 0,
“AssignedQuotaSpace”: -1,
“ReportedQuotaError”: false,
“ReportedQuotaWarning”: false,
“MainOperation”: “Backup”,
“ParsedResult”: “Success”,
“Version”: “2.0.7.1 (2.0.7.1_beta_2023-05-25)”,
“EndTime”: “0001-01-01T00:00:00”,
“BeginTime”: “2024-02-08T14:42:53.7446268Z”,
“Duration”: “00:00:00”,
“MessagesActualLength”: 0,
“WarningsActualLength”: 0,
“ErrorsActualLength”: 0,
“Messages”: null,
“Warnings”: null,
“Errors”: null
}
}

3

Welcome to the forum @dr8876

I only see one backup version here. Is this the initial backup?
If so, what do you do to get this repeatedly? Exact same file?

It looks like your Destination isn’t working properly. What is it?

Verifying backend files tested three files, found one came back corrupted. Other two came back OK.
Might be that the largest one failed, but this is such a small sample that a trend can’t be determined.
The TEST command can check all the files if you like, but let’s first get a preliminary sense of things.

is probably retrying the bad file to see if a good one can be downloaded, but it wasn’t, so a test error.
If problem is reproducible, you can watch About → Show log → Live → Retry to watch its retries live.
Clicking on a problem line will expand with explanation details which might explain the compare error.

4

This looks like a temporary file generated by Duplicati itself while it is running a backup. If this is the case, it may be normal that there are discrepancies between these files and the backup, since they could have changed during the backup.
If I were you, I’d try to exclude all temporary directories from backup since these files are well, temporary and it’s just a waste of time and space to back them up, and it could get you misleading error messages.

this is looking like an open file that is locked, it’s fairly normal to get messages like that. A simple workaround is to run your backup when all files are closed. A more elaborate strategy is to run Duplicati as a service and to set the advanced setting snapshot-policy to ‘On’.

5

The context (lines above that) seems to say it’s from a Test, one of three files in the test.
The timestamp has no time later than it in this log. That’s right where the Test should be.
The form of the message is exactly what BackendManager does after download to Temp:

I’m not sure what E: is, but if it’s the Synology FS by SMB, I don’t think VSS will be able to help with it.

EDIT:

VSS for SMB File Shares is an interesting twist, but it needs OS support that’s likely not available here.

6

Yes, right. It could be the time to dig up this old PR that I fumbled in simpler, happier times. I was expecting it to be a fire (PR) and forget (Duplicati), I was still believing in Father Christmas.

7

There is always debate about how loudly to report various odd situations, as Warning is a bit noisy.

I think there have been at least some forum requests to be able to turn off annoying yellow popups.

The counter-argument is that some people set a high bar on the perfection of the backup, e.g. here:

I didn’t even get into the locked file part, as those are “normal”. I was commenting on original post on

which connects to named Temp file though the Key/Value pair in the Verifications list for the Test.

I don’t know if Windows Temp is also in Source, but if it is, then I agree it probably doesn’t need to be.

@dr8876

What is the E: drive? If it’s the Synology, how is it connected? I had guessed SMB, but it’s not certain.

What is the Destination? It looks like it’s not fully successful at keeping files intact, so needs attention.

9

I understand this. Gotcha. I will just close files when backing up.

10

Thanks so much for your help and the welcome.

This is the initial backup. The computer was recently formatted.

The destination is a synology drive, remote from where I am working currently.

As for everything else, I am a complete neophyte when it comes with Duplicati. I don’t even know how to use the command line feature in Duplicati. You would need to walk me through.

The last time I was using this computer, before I reformatted, I erased both backups, local and remote files. The network Amazon drive that was the source for both of these backups is still intact.

11

I didn’t understand any of your answer relating to the temporary files. Sorry. I am a neophyte when it comes to Duplicati and backup software in general. I just need to have the backup run on the default settings.

12

It is a WEBDAV synology destination.

13

E drive is an Amazon S3. That is the source. The destination is a Synology WEBDAV connection. I have opened the folder on the Synology drive that the files back up to. I see a bunch of .aes files. There is something wrong with these (not intact)? I don’t understand why the backup isn’t working. I did delete the remote files in this folder on Synology when I deleted the backup (before or after formatting this computer, can’t remember), but the files on the Amazon S3 are fine.

14

Thanks. Then we’re looking at a possible WebDAV issue. Do you also have remote shell access?

Do you run something that let’s you treat S3 as files? Whatever it is, it seems to be able to error:

We’re actually not sure it’s a locking problem, and I’m even less certain knowing it’s S3, somehow.

You don’t know which until you test using either test command or an equivalent test using a shell.
Individual files like duplicati-b8d4e45575ba545c1b0c87091cc6fccb7.dblock.zip.aes are feasible to download to the PC to decrypt, e.g. using AES Crypt GUI, or Duplicati’s SharpAESCrypt.exe CLI.

Using the Command line tools from within the Graphical User Interface to convert it to do test all is simpler if you want a big download study, which might take awhile, depending on download speed.

EDIT:

A manual download and decrypt would just be to confirm that there’s really a problem with that file.
IIRC the test that Duplicati does can fail either on the decryption integrity check, or Duplicati’s own.

15

I can access the synology quick connect dashboard, including files. I don’t know what remote shell access is. Sorry.

I run Mountain Duck in order to open the S3 files on my computer.

For the last section of text regarding testing files, I’m so sorry, but I didn’t really understand that. You are talking about downloading the AES files and batch testing them for errors?

16

@ts678 If it helps, I am using Storj, connecting to it through an S3 connection using Mountain Duck. I set up the Duplicati backup so that it sees the Storj drive (S3) as the source and the Synology (WEBDAV) as the source.

When I set up the backup, I clicked the “Test Connection” button, and it said it worked. So I assumed that Duplicati had some level of access to Synology.

EDIT: I set up the Duplicati backup so that it sees the Storj drive (S3) as the source and the Synology (WEBDAV) as the destination.

17

ah, I see, you are right as usual.
It’s utterly misleading and wrong to use ‘tmpfile’ here, it should have been item.RemoteFilename like in the error message above.

18

No problem. Duplicati GUI works fine too, but local test on Synology CLI would avoid file downloading.

Thanks. There are a few options. The more things that get thrown in, the more guessing what did what.

And now I’m seeing your next post that S3 is Storj. Regardless, let’s focus on the possible bad file first.

I talked about testing the suspected bad one manually, then doing a wider survey simply, using the GUI.
If you prefer, you can do small GUI studies, but the file sample is somewhat random, so results will vary.

Tehnically, that’s completely correct (I think). It’s where downloaded files are (causing complaints from people with SSDs and backups on USB drives that we’re needlessly wearing their SSD, but I digress).

Some Duplicati messages are almost like internal hint aimed at the developer. An example is stuff like

There are some messages that could be updated to be more recognizable, rather than very confusing.
The one in question here (and in fact all of the Complete log, I think) is just a JSON dump of the result.

EDIT:

and the nice view above the Complete log is a human-friendly decoding of some (but not all) of it, so possibly the very-hard-to-read complete log could be left as it is, and the friendliness added in decode.
It turns out that those who do databases can actually put the complete log into the LogData table. I did.

19

I have the GUI command line open in Duplicati. How do I test for that file? I read the content from the link you included, but didn’t really see what I needed to do.

EDIT: I can’t post more replies. I need to wait 3 hours. The forum is making me wait. Here is the screen shot of the GUI options. Which one should I click?

image
image1189×759 55.2 KB

EDIT 2: I am running the tool now. I will share the results.

20

Duplicati.CommandLine.exe test <storage-URL> <samples> [<options>]

In GUI terms, it means set Command to test, keep the Target URL. Change Commandline arguments either to empty, or a number, or all (which will be slow). You can’t name a specific file, but all will test that one eventually. If you want a specific file, you need to do it the hard way. Download, use some tool.

21

I raised you from new user to basic user, which may or may not avoid the limit.
I think some of the limits are time based, for example 24 hours after new account.