Invalid header marker

kef · February 15, 2021, 11:12pm

I been seeing the following for the last 10 days. What’s wrong? How do I fix it?

Feb 15, 2021 6:01 PM: Failed while executing “Backup” with id: 9
System.Security.Cryptography.CryptographicException: Invalid header marker —> System.IO.InvalidDataException: Invalid header marker
at SharpAESCrypt.SharpAESCrypt.ReadEncryptionHeader(String password, Boolean skipFileSizeCheck)
at SharpAESCrypt.SharpAESCrypt…ctor(String password, Stream stream, OperationMode mode, Boolean skipFileSizeCheck)
at Duplicati.Library.Encryption.AESEncryption.Decrypt(Stream input)
at Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output)
at Duplicati.Library.Main.BackendManager.<>c__DisplayClass36_0.b__0()
at System.Threading.Tasks.Task.Execute()
— End of inner exception stack trace —
at Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)
at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result)
at Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)
at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

drwtsn32 · February 19, 2021, 4:21am

What back end are you using? What version of Duplicati?

It sounds like the file may be corrupt. Can you manually download the file it claims has an invalid header marker and open it in Notepad (be sure not to save it though)? The first three characters of the file contents should be AES.

kef · February 19, 2021, 9:06pm

The version is 2.0.5.1_beta_2020-01-18.

I don’t know which file has invalid header marker. There is no indication of the filename or path.

The last completed backup job was on Feb. 4, 2021 (and that one has warnings).

Here is the log entry:

· Feb 18, 2021 6:01 PM: Failed while executing “Backup” with id: 9

System.Security.Cryptography.CryptographicException: Invalid header marker —> System.IO.InvalidDataException: Invalid header marker at SharpAESCrypt.SharpAESCrypt.ReadEncryptionHeader(String password, Boolean skipFileSizeCheck) at SharpAESCrypt.SharpAESCrypt…ctor(String password, Stream stream, OperationMode mode, Boolean skipFileSizeCheck) at Duplicati.Library.Encryption.AESEncryption.Decrypt(Stream input) at Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output) at Duplicati.Library.Main.BackendManager.<>c__DisplayClass36_0.b__0() at System.Threading.Tasks.Task.Execute() — End of inner exception stack trace — at Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile() at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend) at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager) at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize) at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext() — End of stack trace from previous location where exception was thrown — at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method) at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter) at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

drwtsn32 · February 19, 2021, 9:31pm

If you have a tab open to About → Show Log → Live → Verbose, it should show you the filename there. Here’s an example of when I manually caused the problem:

kef · February 19, 2021, 11:37pm

This is what I see:

· Feb 19, 2021 6:02 PM: The operation Backup has failed with error: Invalid header marker

· Feb 19, 2021 6:02 PM: Fatal error

· Feb 19, 2021 6:01 PM: Backend event: Get - Retrying: duplicati-b3a1729c77d4849bcab2ee4c3995ab065.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Operation Get with file duplicati-b3a1729c77d4849bcab2ee4c3995ab065.dblock.zip.aes attempt 1 of 5 failed with message: Invalid header marker

· Feb 19, 2021 6:01 PM: Backend event: Get - Started: duplicati-b3a1729c77d4849bcab2ee4c3995ab065.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Backend event: Get - Failed: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Operation Get with file duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes attempt 5 of 5 failed with message: Invalid header marker

· Feb 19, 2021 6:01 PM: Backend event: Get - Started: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Backend event: Get - Retrying: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Operation Get with file duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes attempt 4 of 5 failed with message: Invalid header marker

· Feb 19, 2021 6:01 PM: Backend event: Get - Started: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Backend event: Get - Retrying: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Operation Get with file duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes attempt 3 of 5 failed with message: Invalid header marker

· Feb 19, 2021 6:01 PM: Backend event: Get - Started: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Backend event: Get - Retrying: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

· Feb 19, 2021 6:01 PM: Operation Get with file duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes attempt 2 of 5 failed with message: Invalid header marker

· Feb 19, 2021 6:01 PM: Backend event: Get - Started: duplicati-b7490276a0a724918a36a09f5232d9bb0.dblock.zip.aes (49.93 MB)

drwtsn32 · February 19, 2021, 11:58pm

Ok great, can you download the file manually from the back end server and view it in Notepad or similar? The first three characters should be AES. What do you see?

kef · February 20, 2021, 2:18am

I don’t know what you mean by “download the file manually from the back end server”? What/where is the backend server?

drwtsn32 · February 20, 2021, 4:45pm

What “destination” are you using?

kef · February 20, 2021, 6:21pm

I mapped OneDrive to Z: using RaiDrive (Z:\Backup). I found the file in question in the backup folder. It is empty – 0 kb. Many other files are empty too.

I attached the entire list of backup files.

By the way, thanks for helping me.

(Attachment filelist.txt is missing)

kef · February 20, 2021, 6:29pm

Rod,

The text file I attached was rejected. Please see the attached jpg.

drwtsn32 · February 20, 2021, 6:58pm

In the screen shot I don’t see any 0 KB files. Were those not included in the screen shot, or do you mean if you copy the file to your C: drive it ends up being 0 KB (but shows with a different size in OneDrive)?

If you view the files on the native OneDrive web UI do they show as 0 KB there?

I have never heard of RaiDrive before. There is one other reference to someone using it in the forum. I’m not sure if that added layer is causing some reliability issues or not, but you definitely should not have 0 KB files!

Duplicati does support OneDrive natively, but I personally don’t use it and can’t provide much help with it.

kef · February 20, 2021, 7:37pm

There was a problem with the original screenshot. The capture application did not finish scrolling to the bottom of the window, therefore it did not capture all of the files.

I attached a complete capture. I sorted the files by file size in ascending order, so you can see the empty files at the top.

drwtsn32 · February 20, 2021, 11:00pm

Do those files show up as 0KB on the OneDrive web UI as well? I’m guessing yes, but it would be nice to confirm.

kef · February 21, 2021, 12:03am

Yes they do. I couldn’t perform a scrolling capture of the OneDrive browser window so I pasted a few screenshot below.

drwtsn32 · February 21, 2021, 5:48am

Well, that’s not good. Those files should not be zero bytes.

It’s hard to say if the RaiDrive is the culprit or not, but I think in general you’d be better off taking that out of the loop and just have Duplicati go straight to OneDrive via its native support.

kef · February 21, 2021, 2:30pm

I tried going straight to OneDrive in the past and it didn’t work. It was a few years ago. I’ll try again.

Ken Flinchbaugh

ts678 · February 21, 2021, 3:03pm

I don’t know if that’s an export/import to a new job or convert of old, but current database has history of what your now-zero-length files looked like before. It requires looking in the DB or creating a bug report.

You can sample what a list looks like in <job> → Show log → Remote, find a list, and click the line.

I’m puzzled by how your files from December managed to go empty in February without a date change.
What else might have happened at the time the error began, e.g. was there a rare system restart then?

Duplicati ordinarily keeps a close eye on the destination files with listing, but doesn’t do mass download.
There are Advanced options to disable verification, but eventually a compact will run, as happened here.

kef · February 21, 2021, 4:13pm

How do I delete the current backup and start over? Do I delete the configuration?

drwtsn32 · February 22, 2021, 2:57am

You don’t need to delete the configuration entirely. If you’re willing to delete your backup history, I would do something like this:

Delete all the files currently on the back end (in OneDrive)
Reconfigure the backup job to use the native OneDrive protocol
Delete the local job database (Click the blue “Database …” link in your screen shot above and click the Delete button)
Start the backup operation.

Duplicati will detect the emtpy back and and a non-existent local database and will “start over” from scratch.

Hope this helps!

kef · February 22, 2021, 2:13pm

Before your last reply I deleted the backup configuration and created a new backup. This time I set it up directly with OneDrive rather than the mapped drive. The backup was successful. See screenshot below. I think the issue is closed. Thanks you for help.