System is ubuntu server 16. Duplicati version 2.0.4.5_beta_2018-11-28. Connection tests to my wasabi bucket are successful. Trying to backup ~1.5gb, ~50 files. Seeing this in the logs;
—> (Inner Exception #0) Amazon.S3.AmazonS3Exception: Error making request with Error Code BadRequest and Http Status Code BadRequest. No further error information was returned by the service. —> Amazon.Runtime.Internal.HttpErrorResponseException: Bad Request —> System.Net.WebException: Bad Request
I just realized I’m running a beta version, so I’ll test with a non beta version. Wanted to post this just in case it could help with development.
I doubt encrypting is related to the issue, but if you think it might be could you create a test backup of a single file or two without using encryption and see if it still fails?
For the errors you’re currently getting, are you seeing them on the job “Show log” page or the global About -> “Show log” tab?
Thanks @JonMikelV for the reply. I’m seeing this error in the about “show log” tab, as well as in the log file I setup in /var/log/.
System.AggregateException: One or more errors occurred. ---> System.AggregateException: The channel is retired ---> CoCoL.RetiredException: The channel is retired
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0:
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0
at System.Runtime.CompilerServices.TaskAwaiter`1[TResult].GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0
at CoCoL.Channel`1+<WriteAsync>d__30[T].MoveNext () [0x00196] in <6973ce2780de4b28aaa2c5ffc59993b1>:0 --- End of stack trace from previous location where exception was thrown ---
and this;
---> (Inner Exception #1) System.AggregateException: One or more errors occurred. ---> Amazon.S3.AmazonS3Exception: Error making request with Error Code BadRequest and Http Status Code BadRequest. No further error information was returned by the service. ---> Amazon.Runtime.Internal.HttpErrorResponseException: Bad Request ---> System.Net.WebException: Bad Request
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00064] in <bd46d4d4f7964dfa9beea098499ab597>:0
at System.Net.HttpWebRequest.GetResponse () [0x0000e] in <bd46d4d4f7964dfa9beea098499ab597>:0
at Amazon.Runtime.Internal.HttpRequest.GetResponse () [0x00000] in <6a6b52bcf0a3446783eaf4189c33ceed>:0
--- End of inner exception stack trace ---
I dont believe it’s encryption as a non-encryption job is not completing either. Seeing the “BadRequest” error I’m guessing it maybe wasabi side? I’ve opened a support case with them and provided them with some additional info they requested, so hopefully a resolution will come shortly.
@wasabi-jim has been helpful in the past with Wasabi specific issue, but I’m guessing it’s not specific to Wasabi as I had a similar error with Box a while ago with an older version of Duplicati:
Unfortunately, I don’t have any other suggestions at this time. You could try reading some of the other posts with RetiredException in them and see if they apply to what you’re seeing at all…
Brand new duplicati install, brand new ubuntu install. I tested wasabi and duplicati several weeks ago on a different system and it worked fine. However, it was hardware, and this one is a vm. This is the initial backup, was not working before. I’ve also tried a single file, with same results. My other hardware test machine died so I’m not able to go back and check what version was working with wasabi.
Thanks for your response, I’ll search through the other posts.
Just an update to this thread while working with wasabi support…I’ve found that if I select a folder with multiple files the job never completes. If I select a single file, it works. I’ve also found some things in a pcap that suggest some issues on the wasabi side, but waiting to hear back from them. This is a strange issue that wasn’t present in previous testing.
If a destination issue is suspected, the source configuration is pretty loosely related after source chunks get packed into dblock file(s). You can watch uploads live in About --> Show log --> Live -> Information or higher. Unless you changed –dblock-size you’ll probably see a flow of 50MB (except for the last one) dblocks, with a smaller dindex after each, and a dlist at the end. Possibly you can see retries (use log level Retry instead of Information). If so, there are a couple of controls such as –number-of-retries and –retry-delay that may help.
Thanks @ts678 for the info. I’ll check the items you mentioned, and see if I’m able to what the live uploads. I’m also waiting on the vendor to replicate my setup and see what they come back with. Will report back when I have more info.
After several hours of troubleshooting I’ve determined its an issue with my firewall. Behind my pfsense box I’m unable to complete a backup over 1mb. Removing pfsense and the backups complete. I’m seeing a lot of tcp retransmits. I disabled all hardware offloading, but this didnt help. Anyone running duplicati behind a pfsense firewall that can offer some advice? I am running pfblocker as well as suricata, but I’m not doing any blocking with suricata. No qos, and rules are pretty standard, nothing crazy. Any advice or ideas would be appreciated.