Backups seem to perform properly only once

Support
1

My backup performed properly on the first run. Then I tried it again and got the following error:

  • 2019-06-24 15:33:38 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-20190621T051006Z.dlist.zip.aes
  • 2019-06-24 15:34:19 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-20190624T182734Z.dlist.zip.aes
  • 2019-06-24 15:35:02 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-b123358b97d8f45949635d465d44f1068.dblock.zip.aes
  • 2019-06-24 15:35:43 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-b4883b3079b60432d9516da422e74fd43.dblock.zip.aes
  • 2019-06-24 15:36:24 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-i11c20bd3a29443ef88634fcee0e3ee61.dindex.zip.aes
  • 2019-06-24 15:37:05 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-i458726ae3bc441438fdd0f04f3537b08.dindex.zip.aes
  • 2019-06-24 15:37:46 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-i5a76d1f74ca741a5bdc32ef42d5e2d3c.dindex.zip.aes
  • 2019-06-24 15:38:27 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-20190624T183244Z.dlist.zip.aes
  • 2019-06-24 15:39:08 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-i29f95ca3937a4761b7468b9ec59843a4.dindex.zip.aes
  • 2019-06-24 15:39:49 -03 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-b4883b3079b60432d9516da422e74fd43.dblock.zip.aes

I did try to recreate and repair the database with no luck. What am I missing?

2

These cryptic single-line messages usually have useful longer (still cryptic but at least more informative) messages on the next line, then continuing for awhile. Can you get something better, e.g. from server log About → Show log → Live → Error

Can you describe roughly what “no luck” looks like? While Repair tries (I think) a large variety of things, the Recreate is probably more dependent on what the remote files look like. One-way flow into the database…

3

Extended error log here:
{"ClassName":"Amazon.S3.AmazonS3Exception","Message":"The operation is not valid for the object's storage class","Data":null,"InnerException":{"ClassName":"Amazon.Runtime.Internal.HttpErrorResponseException","Message":"The remote server returned an error: (403) Forbidden.","Data":null,"InnerException":{"ClassName":"System.Net.WebException","Message":"The remote server returned an error: (403) Forbidden.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.Net.HttpWebRequest.GetResponseFromData (System.Net.WebResponseStream stream, System.Threading.CancellationToken cancellationToken) [0x00146] in <d9a31ddea0124811953b9ec0fbe9be3d>:0 \n at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <d9a31ddea0124811953b9ec0fbe9be3d>:0 \n at System.Net.HttpWebRequest.GetResponse () [0x00016] in <d9a31ddea0124811953b9ec0fbe9be3d>:0 \n at Amazon.Runtime.Internal.HttpRequest.GetResponse () [0x00000] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233079,"Source":"System"},"HelpURL":null,"StackTraceString":" at Amazon.Runtime.Internal.HttpRequest.GetResponse () [0x0003a] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.HttpHandler1[TRequestContent].InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00073] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.RedirectHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00000] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.Unmarshaller.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00000] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00000] in <33736e93ce0247419ab9e4a00562eb53>:0 \n at Amazon.Runtime.Internal.PipelineHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x0000e] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 \n at Amazon.Runtime.Internal.ErrorHandler.InvokeSync (Amazon.Runtime.IExecutionContext executionContext) [0x00000] in <6a6b52bcf0a3446783eaf4189c33ceed>:0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:“AWSSDK.Core”,“Response”:{“StatusCode”:“Forbidden”,“IsSuccessStatusCode”:false,“ContentType”:“application/xml”,“ContentLength”:-1,“ResponseBody”:{}}},“HelpURL”:null,“StackTraceString”:” at Duplicati.Library.Main.BackendManager.GetForTesting (System.String remotename, System.Int64 size, System.String hash) [0x00065] in :0 \n at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x0042f] in :0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“errorCode”:“InvalidObjectState”,“errorType”:“Sender”,“requestId”:“E940995F59212C46”,“statusCode”:“Forbidden”,“AmazonId2”:“9uzTsblAJ9t0RCJf+fUtf/nc9PJMmioSgy+6E026IPxYIYLgiAv1qfV1UOiiax79gHVbYABwm8g=”,“ResponseBody”:null}`

4

Is that lines under an “Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file”? If so thanks. If not, are you sure it matches the original issue? New log might be:

The Operation is not valid for the object’s storage class

and if you want better information, and know S3 facilities and config (I do not), you can do a web search. What I found when searching that error message was usually people talking about Glacier, as above did.

5

@ts678 thanks a lot, yes it seems the issue is because Iḿ using glacier. I’ve added the flags:
–no-backend-verification=true
–no-auto-compact=true
And it seems to be working like a charm now.

6

I personally would be nervous about never verifying the backup files, especially if the backup is important. Duplicati sometimes has issues (look in the forum), so I hope your backup files are fine if you need them. Having said that, it’s good to hear that having it be “blind” to what it actually has available looks OK so far.