Backups corrupt. How to repair? How can it happen?

Support
1

I have 4!!! corrupt backups (out of 9 tested until now). AGAIN. It is the third time. The first 2 backups were on hubiC. Now it is a small backup on Dropbox, one on hubiC, one on box and one on a local USB-Drive I loose my confidence in Duplicati.

First: Dropbox

I downloaded the remote backup and made a restore of version 1 and got this:

Operation Get with file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes attempt 1 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (9,95 MB) ...
Operation Get with file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes attempt 2 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (9,95 MB) ...
Operation Get with file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes attempt 3 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (9,95 MB) ...
Operation Get with file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes attempt 4 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (9,95 MB) ...
Operation Get with file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes attempt 5 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (4,39 MB) ...
Failed to patch with remote file: "duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes", message: Invalid header marker => Invalid header marker
  Downloading file (3,39 MB) ...

I made a binary compare of the local copy with the one on dropbox. The whole backup is binary identical.

The end of the restore shows

  Downloading file (204,31 KB) ...
  Downloading file (5,10 MB) ...
  Downloading file (3,45 MB) ...
  0 files need to be restored (0 Bytes)
Verifying restored files ...
Restored 616 (180,87 MB) files to z:\temp\restoreV2
Duration of restore: 00:01:59

So I have 616 files restored. One backup file has an invalid header marker but there is no information what this means for the restore process. Are some files not restored? Are some files restored in an older version?

Seconds conspicuity: If I list version 1 I get the information that the version has 615! not 616 files like restored:

Z:\temp\dropbox-01_HASHTEST>"c:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" list .\ --version=1 --dbpath=C:\Us
Listing filesets:
1       : 23.02.2018 07:58:48 (615 files, 180,87 MB)

If I list version 1 with * for all files I get also 615 and not 616 files. So one file was restored but is not shown in the list? How is that possible?

list-broken-files shows everything is ok:

Z:\temp\dropbox-01_HASHTEST>"c:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" list-broken-files .\ --dbpath=C:
No broken filesets found in database, checking for missing remote files
  Listing remote folder ...
Skipping operation because no files were found to be missing, and no filesets were recorded as broken.

Second a hubiC backup with 28 GB (15 MB data files), checked with verify and parameter “all”:

...
  Downloading file (22,03 KB) ...
Operation Get with file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes attempt 1 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (22,03 KB) ...
Operation Get with file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes attempt 2 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (22,03 KB) ...
Operation Get with file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes attempt 3 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (22,03 KB) ...
Operation Get with file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes attempt 4 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (22,03 KB) ...
Operation Get with file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes attempt 5 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
Failed to process file duplicati-i68ec60a4bc9e4893860028dd0fb898ae.dindex.zip.aes => Die angefragte Datei existiert nicht
  Downloading file (877 Bytes) ...
  Downloading file (9,04 KB) ...
  Downloading file (12,58 KB) ...
  Downloading file (9,45 KB) ...
  Downloading file (8,11 KB) ...
...
  Downloading file (14,95 MB) ...
Operation Get with file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes attempt 1 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,95 MB) ...
Operation Get with file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes attempt 2 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,95 MB) ...
Operation Get with file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes attempt 3 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,95 MB) ...
Operation Get with file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes attempt 4 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,95 MB) ...
Operation Get with file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes attempt 5 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
Failed to process file duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes => Die angefragte Datei existiert nicht
  Downloading file (14,97 MB) ...
  Downloading file (14,94 MB) ...
  Downloading file (14,93 MB) ...
...
Operation Get with file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes attempt 1 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,90 MB) ...
Operation Get with file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes attempt 2 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,90 MB) ...
Operation Get with file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes attempt 3 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,90 MB) ...
Operation Get with file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes attempt 4 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
  Downloading file (14,90 MB) ...
Operation Get with file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes attempt 5 of 5 failed with message: Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht
Failed to process file duplicati-ba707c45d4ec5460ba53682c4d0de2edf.dblock.zip.aes => Die angefragte Datei existiert nicht
  Downloading file (14,96 MB) ...
  Downloading file (14,97 MB) ...
  Downloading file (14,99 MB) ...
...

still running but index files are missing and data files are missing.

The hubiC space is only used by Duplicati. No other programm has access and also do not access it by web access.

Third the local USB-Backup I have this verification error

  Downloading file (499,98 MB) ...
  Downloading file (499,99 MB) ...
Operation Get with file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (499,99 MB) ...
Operation Get with file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (499,99 MB) ...
Operation Get with file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (499,99 MB) ...
Operation Get with file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (499,99 MB) ...
Operation Get with file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content

There was no password change. This filed was backuped up continous as you can see on the timestamps:
image

Fourth: box cloud. Also error with password in about 40!!! Files and Header error in some other files??? Downloaded so of the files again and compared them. No download problem. Files matched:

...
  Downloading file (14,92 MB) ...
  Downloading file (14,93 MB) ...
  Downloading file (14,93 MB) ...
  Downloading file (14,93 MB) ...
  Downloading file (14,94 MB) ...
  Downloading file (14,91 MB) ...
  Downloading file (14,95 MB) ...
  Downloading file (14,92 MB) ...
  Downloading file (14,95 MB) ...
  Downloading file (14,93 MB) ...
  Downloading file (14,91 MB) ...
  Downloading file (14,95 MB) ...
  Downloading file (14,97 MB) ...
  Downloading file (14,93 MB) ...
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,93 MB) ...
Failed to process file duplicati-bab7462f605304ad8bf29f59a16607471.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,94 MB) ...
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-bac4b1e25b48d4cb0a430cde90e5032b8.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,96 MB) ...
Operation Get with file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,96 MB) ...
Operation Get with file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,96 MB) ...
Operation Get with file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,96 MB) ...
Operation Get with file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,96 MB) ...
Operation Get with file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-babfa94f297584ea8b41481f7cdc2c687.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,91 MB) ...
Operation Get with file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,91 MB) ...
Operation Get with file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,91 MB) ...
Operation Get with file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,91 MB) ...
Operation Get with file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,91 MB) ...
Operation Get with file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-bac94aac9a639408686a76b3e17611bb3.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-bad48a8c0df5340b3b23451db813243ac.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,98 MB) ...
Operation Get with file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes attempt 1 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,98 MB) ...
Operation Get with file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes attempt 2 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,98 MB) ...
Operation Get with file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes attempt 3 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,98 MB) ...
Operation Get with file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes attempt 4 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (14,98 MB) ...
Operation Get with file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes attempt 5 of 5 failed with message: Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
Failed to process file duplicati-baeb3779e25564d58b1dd9bf27b296c08.dblock.zip.aes => Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content
  Downloading file (15,00 MB) ...
...
Failed to process file duplicati-bd3160fb5ec674ca29151626dfcd85c5a.dblock.zip.aes => Invalid header marker
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes attempt 1 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes attempt 2 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes attempt 3 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes attempt 4 of 5 failed with message: Invalid header marker => Invalid header marker
  Downloading file (14,92 MB) ...
Operation Get with file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes attempt 5 of 5 failed with message: Invalid header marker => Invalid header marker
Failed to process file duplicati-bd335faf5aed0436ea9c593e2fc443ea2.dblock.zip.aes => Invalid header marker
  Downloading file (14,91 MB) ...
...

I tested 9 backups (4 local, 2 dropbox, 1 google, 1 hubic, 1 box) and 1 local, 1 dropbox, 1 box and 1 hubic are not VALID and I have 3 DIFFERENT ERRORS.

And an update. Through the night a verify was running with a fourth type of error after I tried to repair the local copy (purged missing files and compacted, lost about 3 GB of backup data) and after that I got this in the next verify run:

duplicati-i09b06d5f94c9499b951ef3efee17dee9.dindex.zip.aes: 231 errors
        Extra: ++wm+4zckBAdHA7CmTlIQryZCiARWNWQJ8uV1AI15a4=
        Extra: +1rxoqQHHGGnaLoXeIPNizkJwdTBQG0D+bPrRiddw6k=
        Extra: +88ktIZ7Rg+Q9ZcD2yKh7MZebvdS8qX13a5oP3/FcNo=
        Extra: +M+kApPYv6Dn825WoeI5LmYbps/5lDs5DctsJ0Sk08Q=
        Extra: +Ta1Pbe+OllErqqjGeklDKkDFayYaLW928q9diajB5Y=
        Extra: +UDT87KLfYo3qLw4gzBPqyPX90FyhA70M5s8ehaQgD4=
        Extra: +Z2l0yfRqAlwPhTNyGX+Cr1z8JxkgoU6+ieuGPcal+U=
        Extra: +ygUSu21/+IMlD02fKdP3TIaJzWd07WA3bR0BtYjlvw=
        Extra: /D+GBeIYpP/oWJ52SWQ24DnLrE+kKq4mZl+yNSPrRxY=
        Extra: 0PzoO8YpHtY2ezp1RQoHfYmqoBngiu2v5EHPPcoDB6M=
        ... and 221 more

duplicati-i1a80950bbe894bada039e09de83c406e.dindex.zip.aes: 366 errors
        Extra: +DInLAcULQ0vl6THYJx9P/qs76jUnRoT93ohxleCn7A=
        Extra: +GT/LtND4iOBF0TeVployBw2wy1aeknFBKMJQPFHPGU=
        Extra: +WToBY5GHcDqLmwlcKg+384gfa8eH76ZaXdJqgqcEoY=
        Extra: +ZsPLlnq9XsGS9QElZ2snzwkOojkojXxIGX5xbT4TJo=
        Extra: +dDXLdWa/G8Dw9gFV9tE987KkP/UduROqxNUSu/Oktw=
        Extra: +gUmtcfnW+IJuN1PtcCkZ7+OmbN5iG9QyxKUBeCVyYw=
        Extra: /2Inq3ughHFYo3BA3o8a5rXj4Yz8Jlr2FyqacuUBLIY=
        Extra: /EOcj50yQOy/cK0vm6YAqioE7DN66b7frs2zOxH9X+Y=
        Extra: /PY/D6CZ+qEnMBx8AvsWbhVpzK6Eroj4hnIuY2O5FnI=
        Extra: /aHnN9lKZU+uHjjNSVOROOh2TP+Q0wAAndnzuzCQaWs=
        ... and 356 more

duplicati-ia99a5b7f77164687a10f507e208af137.dindex.zip.aes: 529 errors
        Extra: +/OV0qexHY3UyGoyLHCLAvdfy5LxJ7utaBjTPjvH55A=
        Extra: +55/6jyLbjYApFperxGIxBNvCEWLR0mLt9wKwKzBYU8=
        Extra: +AwUO7fEgRDEYJmUeyn8pQtBnbiqZJCFv3xpUcmSvIw=
        Extra: +IRt/guNtrmL+UOWonetuv+x+xUOR/WqwwOB+lBFsZI=
        Extra: +Lb4aYZdp3Y2TE9bUpxrfjXtOOZ4Mj4KDdoi+92ic4c=
        Extra: +P6RlB3JukC6GkFaQVrTBACJ3U3yi8hW3y7UtaTNjpc=
        Extra: +T1sgiHLGZ0ylUR1TQ7G99m8Gt6vBvXr3M3Dv9m87/A=
        Extra: +nuuDZsmJ3g6diXiTML+/gPxAPLlEOFRL0S1r3lDnU0=
        Extra: +tAGOsxOHUlD2+wwyN2IvI33jvpMlUqxzz2uo8CzaqI=
        Extra: +wj+GaR3KbEO8QfxuJEa4IhDBtZb5I3DyNK9otYtRxM=
        ... and 519 more

duplicati-if9b1d8c6c3b24ec4af6b5c672e5eb357.dindex.zip.aes: 154 errors
        Extra: +0KbR2e1vqXjPuf+FBwTQJDtK4AqcepBUn12NCUUxw8=
        Extra: +8dSVkffU9K3lDfv08cEjCj4ZEVVvn64h5sXpZoFUTg=
        Extra: +QR3FSvTeaM6iAo7j/fOsc8ZWXG7gBfMg36B6l554NU=
        Extra: +dobI5wZFyQIBOe0iz3AF0CHD9hRMmNL/BIjjQv4DNQ=
        Extra: +v+4PBjjFc7qLiACa0DgBeEzmSqwITyCwGk+BBjk9M8=
        Extra: /yRbilOTFR0cSdF6pwMRpHn0Lkx8eG1AabYqSE5bgLY=
        Extra: 1T7dEKbDW9Lw9o/0KNdEoXh0ZeO63Z4V5g+jt3ztM64=
        Extra: 1Zv928pjaVicvILKNtxx9eHjN6Va0PqnFPweCHTmUOo=
        Extra: 27YnI4+L4MtmCzmZjvu4k4k6YGLmqck3XcVHtTlRM7E=
        Extra: 2wZlL0F/7ZkZ1nITj5qtVuEM/ycOiWS8+GC020CUec0=
        ... and 144 more

duplicati-i8595a0e76ecd49aa90f605fd225eb48c.dindex.zip.aes: 404 errors
        Extra: +5k/1c+PhguTG9ujdfOPQuJ02NoaJ5t84KXfQqcGIZE=
        Extra: +UBpqTPn/IzascwXAcq575isyjaFita3SYVcN/7EGy8=
        Extra: +hOpwIAV66a2Wybt1VqY7p3mjt5MHJx3SmtXYB3BZ7I=
        Extra: /K9GlU2THNynacS633gfAt8eM4PyHRMOt+HRVr6KiEc=
        Extra: /L6KreKQQK8U2bqiIdq2Ven42Z6qZBvQFtnA6rzD6Dk=
        Extra: /LvGBIdS4pwlVuoRxMELZjVY0WvXniY3ioYICK3VH/4=
        Extra: /fr5Y7awGrcR2U623YOBHnZieQ4EWpUmxqKoHCGuQa0=
        Extra: /pTftSBPfQgxPhSl3VDYhC7Sogk6+53Qe971fBPzhfE=
        Extra: /yvRcp0NRFtpIVjEVidje/FtJ3j3+Aj8La/m0+NvaKQ=
        Extra: 0GcriFMcYHilIUPXt+ZhjNIXoFJFpDtoHtnd79+pPeo=
        ... and 394 more

duplicati-i026a644b5c504f5ebd23740bb6aecaa1.dindex.zip.aes: 184 errors
        Extra: /hnedK2RJUjhozBpP7n/0hOe0OuREAK6X7zXUzJfc64=
        Extra: /kQc2WQbCFTb8LiQ/rT1VJhsLiq/vN8ALH4EKujjBeQ=
        Extra: /kysCO4JR6Wla0JvRS+sbv48RKIa/RfGsR6YuCxEfkU=
        Extra: /tJUVCyNvI6Hobtc09y7sqYSwHEHrjq5u984Kp4EMVk=
        Extra: /wRxGauYssnmrjT4W9YeBFY+KZ9p+x5h+5j/0kz4qRs=
        Extra: 0BIosy7WKSlesSFQCxspPkVkDFvHCS+sZXS/+gshcWk=
        Extra: 11ufw2VQ2ryQS4iPlBoXmtcWKJBry9HMTAoybFcgE5k=
        Extra: 195XkfH+W5mEINWe2hnJUt13r780KNQzsUBwv/di1og=
        Extra: 2+vlbDj3/XKdLE5fHDWXvaqXP3udi2JYVT08QhSD2lM=
        Extra: 2jGrt1Cerrbbuiw4ju/2B9X7FcsR9cXXM99XRlOKK5M=
        ... and 174 more

How can this happen? First I thought is can be a problem of hubiC but now I have also a corrupt header in the Dropbox and a invalid file in a local backup.

How can I fix this errors?

2

There are two things that came into my mind:

  1. Are you using some specific settings?

  2. Are you sure that your main machine is working properly? If there is e.g. a hardware issue with the RAM, this can cause some really weird issues.

3

That is a lot of problems in one go!

Lets take them one at a time.

USB backup

This is what is probably simplest to investigate, and also the most concerning.

This means that the HMAC integrity check fails. The file looks correct, but something is not right with the file contents. If you have the database that was originally created with the backup (i.e. not from a recreate/repair command), can you look in that and see if the SHA256 hash of duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes matches the file on disk?

If the two hashes match, then it indicates that Duplicati has written a file that it cannot read itself, which is a grave problem that needs to be investigated. Any further information that you can give on this is much appreciated.

Dropbox

This means that the data returned from Dropbox was not the file (it should start with the AES header marker, and it does not. I do not recall seeing this error with Dropbox before, but there could be a number of reasons, such as temporary glitch on Dropbox, error in the downloader, etc. If you look at the file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes with a hex editor, the first 3 bytes should be be the characters AES. I am very curious to hear what are in those three bytes.

That does sound strange. My best guess is that the LIST command is excluding the source folder itself. The Restored files counter is applied after restoring all files and when checking the file integrity. The LIST counter is querying the database table. One potential source of this could be that some file is “restored” twice, but I have not seen this before.

Can you try this and see if it outputs 616 or 615 lines:

"c:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" list * --version=1 --full-result --dbpath=C:\Us

HubiC

Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht

That indicates that the file was uploaded at some point, but now it does not exist. Can you see the duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes file on HubiC via the webinterface? Can you download it?

USB Passphrase error

That error message means that the file appears to be valid, but does the passphrase was not. If you have only a single file with this error, I am leaning towards some memory corruption as a possible explanation.

Can you try to decrypt the file duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes with AESCrypt:

Box.com

That looks like a mix of the other problems, I think once we figure out what happens with the others, this will reveal/fix these as well.

I am not 100% on the context, but it looks like the data files are perhaps broken somehow (from some of the other errors?), and then the verification step sees these files that are supposed to be there, but not hence the Extra: problems.

I hope we can figure out what happens, and then come up with a solution. Luckily it seems you are only testing this, so no real data has been lost.

4

The backups were made with 2 different machines. The source-drives are about 6 or 8 disksI use complete default settings with the exception:

  • Backups to cloud use 15MB, Backups to network drives use 500MB blocks
  • Some backups use “–check-only-filetime” because there are 100K+ Files which never change (SVN repository).

That are the only things I change if I start with a new backup job.

5

OK, we should look into the details as Kenneth suggested and go through them one by one.

6

SHA of the file

Ok. I does not match. For 1 file I can say “Ok. maybe something went wrong”. But in the box.com backup I have > 50 files with this error. But I do not have the original database. I have made a repair at some time.

The file does not start with AES:


type \\duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes
Cg¯+→Ú§ð8jj╣þ■£─°®Ë├ˤ&èí&╗üº☺¸╩Í─%ö┌ Oýª░FH ╚LH¨w¬Iüèýg♫/╔½÷─ñu← Û/eªq┤▼zËı<♦þÇ·ï[â0»ì@ô╣§O,R╝Ë÷¹| æ▼¶╩925▲┘ySÃb¼▲■d"┐Ë#

It is not the source directory. I have removed all directories from the output and got 615 files but there were really 616 files restored!

I think this is really a problem of hubiC!!! The files (6 files in sum) are not visible in the web tool BUT are visible when using CarotDAV but if I try to download with CarotDAV I get 404 not found. Also the hubiC-Sync tool I started downloads everything but not this 6 files and shows error that this files does not exist.

=> so hubiC IS A NO GO!!! File visible in 2 tools but not downloadable. So I think this is really a data loose on the side of hubiC

I have over 40 password error files (then I stopped) The files are 15MB. If that would be a defect RAM I think I would not be able to work with the computer any more.

AND: I have made a Windows Memory Diagnostic on both computers without a problem.

Thank you for you help.

To shorten this: I will start over with my backups and avoid hubiC as Cloud provider!

I will then save the “original” database and after backing up some days I will make verifies.

7

In simple words: The file was give to the USB drive, we stored a hash or the file, and when the file was required, it was broken and had a different hash. This means the file has been modified in the meantime. Is that a hint that we should look into the way duplicati tries to verify backup files every now and then? Shouldn’t the verification routines have found this error (sooner or later)?

Different backend, similar problem: The file has been modified in the meantime. Again: Shouldn’t the verification routine stumble acroos those cases at some point?

8

In the box.com backend there was not just one file with this error message. When I stopped the verify there were over 50 files (total 400 blocks each 15 MB)! There are only 6 Gigabyte of backup data and I stopped after 50 error files 15MB each. Valued about 20%-30% of the files were corrupt.

I had hash-erors as you can see here. Is this an indicator?

To be sure my computers, harddrives aso. are ok I started some tests:

  • As I wrote before: Memory Check on 2 computers without error

  • I have Gigabytes of RAR-Files which my server produces every night (the computer duplicati is running). The same files that go into the cloud with duplicati go also to an external drive with RAR rotating in 4 week-folders. The last hours I tested about 80 Gigabytes without a CRC-Failure.

  • I also have direct 1:1 copies from one computer to another. I compared all files that have changed the last 21 days. Also compared many, many Gigabytes. No binary difference.

  • If one of my harddrives would have a defect there won’t be a problem with backup to USB-Drive AND backup to box.com AND backup to dropbox.com.

So I have

  • no problem with robocopy direct 1:1 copies over local network, the files are binary identical
  • no problem with rar copies over local network, the rars are CRC valid

So it is a little bit strange that the only problem I have with data corrupting is with duplicati backups. But I must say: Duplicati is the only application I use that uses cloud-providers!

At the moment I donwload a mega nz Backup with about 10GB. I will verify that in the evening where the download is ready.

9

Great, that you spend some time to look into this. When you talk about binary differences: Can you say where the differences are? Are they at the beginning of the files? At the end or at random positions?

On the one hand, it is difficult to believe that cloud storage providers are that bad. On the other hand, it is also difficult to believe that Duplicati writes bad files.

It would really be helpful, if Duplicati could be more careful and verify more files each time it runs. I’ll discuss with Kenneth if we have a whole in the verification process, so that backend issues remain undetected or if we can add some cheap verification that finds more basic errors.

10

I think that makes sense.

Might be tricky to make cheap checks since we can’t check the files without downloading them.

Although maybe we could have Duplicati periodically asking to download each file on the backend and then exiting once the backend starts to deliver parts of the file. That should find the above issue with files not being downloadable even though the backend says they are there.

11

The verification actually downloads some files and checks if those are OK. At a rate of 30% corrupt files, this should have found something. I wonder, why itdie not (this is the whole that I mentioned).

If the files are modified in the first bytes, we can download just the header of files to check if the files are present and the header is un-modified. Just some ideas.

12

Is that supported on all backends? Or can we cheat because the backends always deliver the file header first when sending a file?

13

This is a bit of cheating. You request to download the file and stop downloading after a few bytes. It gives a confirmation that the files is definitely present and you can check a few bytes. I am not sure how likely it is that a corrupt file is corrupted at the beginning. I hope to get some feedback from @thommyX about that.

14

Ah, good. It does work like that.

I think it would be ideal to have a check like that periodically on all files. That way at least we know the file can be retrieved and isn’t corrupted in the header.

15

That is exactly what the verification is supposed to do. We must check why it did not detect all those problems.

16

Oh, I thought it only checked a few files each run?

17

Yes. Just a few files. But if 30% are broken, it statistically should have detected a problem after the third run. Obviously, it did not.

18

Did @thommyX mention how many backup runs he did on each backup job? Above it seems he always tries to restore version=1, so perhaps each was only run once.

If there was only 1 backup run would it even have verified anything?

19

You misunderstud me. The binary comparissions of my robocopy backups were ok! NO DIFFERENCE.

Also I compared the files with error the I have downloaded locally for verify check again with the could using FC-Command and CarotDAV as local webdav for hubiC, box.com and dropbox (FC /B for single files, otherwise I use FreeFileSync which has various comparisson methods, by time, by size, by content aso).

20

The box.com Backup run often!

image
image1337×527 30.2 KB

Sometimes there was an error but most runs were ok:
Like this one:

Failed to process file duplicati-bd290e2fc113a43e58808101d783423dc.dblock.zip.aes => Hash mismatch on file "C:\Users\server\AppData\Local\Temp\dup-a03bebdd-03f0-4da9-b3b1-be88b3cf0954", recorded hash: c7AK4Srd1adnIaW8/TXJqOEOjQQKJoOhv9hxf+CgBC8=, actual hash UiIKGYWhof4Iz+ebk9HW9lGmDT8KVl8e3VzomVlrtV0=

Or this one

Failed to process file duplicati-bafa244e861dd4646b7b7412cfb420874.dblock.zip.aes => Hash mismatch on file "C:\Users\server\AppData\Local\Temp\dup-98c9c1c5-8d10-49d5-8a70-9fa917959ad4", recorded hash: Z1F1nTtC5vRRJkPeyYjzBwqY2Qbm+KRvS6As430Hk5Q=, actual hash TV2VDMypRsVVp+cP8Ky1NQ7uv498KJE/J43lMsi90qc=

That was the reason why I asked for this hash failures in the Thread. Diffent backtup but the box.com had also this Hash Errors:
Help: Hash mismatch error