Backups corrupt. How to repair? How can it happen?

[thommyX]

@kenkendk

USB backup

This is what is probably simplest to investigate, and also the most concerning.

thommyX:

Fehler beim Entschlüsseln der Daten (ungültige Passphrase?): Message has been altered, do not trust content

This means that the HMAC integrity check fails. The file looks correct, but something is not right with the file contents. If you have the database that was originally created with the backup (i.e. not from a recreate/repair command), can you look in that and see if the SHA256 hash of duplicati-b6320983bc03b40188d8dbcfb8283daec.dblock.zip.aes matches the file on disk?

SHA of the file

Ok. I does not match. For 1 file I can say “Ok. maybe something went wrong”. But in the box.com backup I have > 50 files with this error. But I do not have the original database. I have made a repair at some time.

@kenkendk

Dropbox

thommyX:

Invalid header marker

This means that the data returned from Dropbox was not the file (it should start with the AES header marker, and it does not. I do not recall seeing this error with Dropbox before, but there could be a number of reasons, such as temporary glitch on Dropbox, error in the downloader, etc. If you look at the file duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes with a hex editor, the first 3 bytes should be be the characters AES. I am very curious to hear what are in those three bytes.

The file does not start with AES:

type \\duplicati-b8903ffb010ac41c79fdb5eebff19b4f8.dblock.zip.aes
Cg¯+→Ú§ð8jj╣þ■£─°®Ë├ˤ&èí&╗üº☺¸╩Í─%ö┌ Oýª░FH ╚LH¨w¬Iüèýg♫/╔½÷─ñu← Û/eªq┤▼zËı<♦þÇ·ï[â0»ì@ô╣§O,R╝Ë÷¹| æ▼¶╩925▲┘ySÃb¼▲■d"┐Ë#

@kenkendk

That does sound strange. My best guess is that the LIST command is excluding the source folder itself. The Restored files counter is applied after restoring all files and when checking the file integrity. The LIST counter is querying the database table. One potential source of this could be that some file is “restored” twice, but I have not seen this before.

Can you try this and see if it outputs 616 or 615 lines:

It is not the source directory. I have removed all directories from the output and got 615 files but there were really 616 files restored!

@kenkendk

HubiC

Die angefragte Datei existiert nicht => Die angefragte Datei existiert nicht

That indicates that the file was uploaded at some point, but now it does not exist. Can you see the duplicati-b475693647b054408a162c6542fe2ec80.dblock.zip.aes file on HubiC via the webinterface? Can you download it?

I think this is really a problem of hubiC!!! The files (6 files in sum) are not visible in the web tool BUT are visible when using CarotDAV but if I try to download with CarotDAV I get 404 not found. Also the hubiC-Sync tool I started downloads everything but not this 6 files and shows error that this files does not exist.

=> so hubiC IS A NO GO!!! File visible in 2 tools but not downloadable. So I think this is really a data loose on the side of hubiC

@kenkendk

Box.com

That looks like a mix of the other problems, I think once we figure out what happens with the others, this will reveal/fix these as well.

I have over 40 password error files (then I stopped) The files are 15MB. If that would be a defect RAM I think I would not be able to work with the computer any more.

AND: I have made a Windows Memory Diagnostic on both computers without a problem.

Thank you for you help.

To shorten this: I will start over with my backups and avoid hubiC as Cloud provider!

I will then save the “original” database and after backing up some days I will make verifies.