Sorry to be asking about B2, but I just can’t get it working with Duplicati 2.0.4.5_beta_2018-11-28 on Fedora. (And I’ve searched, e.g., First attempt at B2 cloud storage backend fails. What to put where? In my case, the failure occurs at “Test Connection,” with the error message “Failed to connect: Error: TrustFailure (Authentication failed, see inner exception.)”, but I don’t know where to find the “inner exception” (can you tell me?). I’m using a B2 AppKey, so the connection URL looks approx like this (yes, I’ll change it now): b2://FooBar/?auth-username=002ba0829d05f780000000002&auth-password=K002Gn12smVEdjlv54z0%2FnaRD6q2GYQ.
See my post here to make sure you’re entering the right information in the correct fields
Yes, thank you, I had seen your post, and I think I’ve tried all these. For example, looking at the example I gave: The Bucket Name is FooBar (I’ve tried both with and without a subfolder, such as “/backup”, nothing works). The auth-username (a.k.a. B2 Account ID) is 002ba0829d05f780000000002. The auth-password (a.k.a. Applicaion Key) is K002Gn12smVEdjlv54z0%2FnaRD6q2GYQ (where the “%2F” is the URL encoding for “/”).
I was wondering if that breaks something. My keys don’t have any /. I was hoping your next key wouldn’t, then we’d know that there’s a problem when B2 issues a key that has characters that need encoding…
For the most complete logging you can set –log-file (or maybe server logs at About → Show log → Live → Warning will be enough to see the Inner Exception), but logging of activity to the remotes is a little thin, including the authentication (where capturing too much information in the log would be a security hazard).
Just to be clear: It’s my understanding ALL B2 AppKey’s (not the Master Key) have a slash in them (according to drwtsn32 at Backblaze which number is which). The encoding “%2F” only shows up in Duplicati when Adding Backup, if you click on the 3 Dots at the top of the dialog and Copy Destination URL to Clipboard. Note that it’s B2 who generates the KeyID/AppKey pair, not me. Thank you for the About --> Show Log --> Live, I’ll try it now.
EDIT: I just tried the logging, but it doesn’t help. Just a string of msgs (at profiling level) like this:
- Jul 9, 2019 3:26 PM: Operation List with file attempt 4 of 5 failed with message: Error: TrustFailure (Authentication failed, see inner exception.)
- Jul 9, 2019 3:26 PM: RemoteOperationList took 0:00:00:32.085
- Jul 9, 2019 3:25 PM: Backend event: List - Started: ()
- Jul 9, 2019 3:25 PM: Starting - RemoteOperationList
- Jul 9, 2019 3:25 PM: Backend event: List - Retrying: ()
- Jul 9, 2019 3:25 PM: Operation List with file attempt 3 of 5 failed with message: Error: TrustFailure (Authentication failed, see inner exception.)
If you mean the Example, that was an example. The slash was in a different spot, and my keys have none.
If @drwtsn32 actually got one with a slash, then perhaps some testing with it could be done. I have no “/”.
Or this theory might be wrong, but I don’t know what else would explain the seeming authentication failures.
EDIT:
That looks like the live log. Sometimes one can click on lines to get more. I forget if these sorts will do that. Actual logging to a file might get a big stack trace, but I’m not confident it will lead right to understanding…
Thanks for trying to help. I think the “/” might be a red herring. For, I just now tried using the B2 Master Key (instead of a subsidiary AppKey), which does NOT have a “/” (or any other strange char, so the URL is the same as the key, no special encoding), and it’s failing too, in the same way. It’s frustrating, because the Bucket certainly exists, and the KeyIDs/AppKeys certainly exist, and access rights all seem to be right, and my account/billing info is all right, and I can manipulate files in B2 via its browser/web interface. It’s only Duplicati that’s failing. FWIW, I’ve only ever used Duplicati with SSH previously, never with one of its canned cloud storage providers, maybe I should try with a different provider (not B2), just to see if I can get one of those working.
EDIT: About that log, you’re right, it was Live, and I now see that clicking on the lines gives more info. (Didn’t know about that before, because never had troubles before, sigh …)
When you switched to the Master Key, did you do the corresponding necessary switch to Account ID?
Yes I did, thanks for the reminder.
I do have a literal slash in mine and I entered it exactly that way… Did not escape it to it’s HTML equivalent.
Well, clicking into that log sure gives a log of info, see below. The thing that jumps out at me is the OpenSSL Cert Failure, near the top. Any idea what that’s about? I’m not knowingly using OpenSSL Certs, so I guess Duplicati’s B2 implementation is doing it.
Jul 9, 2019 3:33 PM: The operation Backup has failed with error: Error: TrustFailure (Authentication failed, see inner exception.)
{“ClassName”:“System.Net.WebException”,“Message”:“Error: TrustFailure (Authentication failed, see inner exception.)”,“Data”:null,“InnerException”:{“ClassName”:“System.Security.Authentication.AuthenticationException”,“Message”:“Authentication failed, see inner exception.”,“Data”:null,“InnerException”:{“ClassName”:“Mono.Btls.MonoBtlsException”,“Message”:“Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED\n at /builddir/build/BUILD/mono-5.18.1.3/external/boringssl/ssl/handshake_client.c:1132”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00054] in :0 \n at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x000c6] in :0 \n at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)\n at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in :0 \n at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0012a] in :0 \n at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x000a4] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:“mscorlib”},“HelpURL”:null,“StackTraceString”:” at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00336] in :0 \n at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x0018c] in :0 \n at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001f5] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233087,“Source”:“mscorlib”},“HelpURL”:null,“StackTraceString”:” at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x00275] in :0 \n at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x0015b] in :0 \n at System.Net.WebOperation.Run () [0x000b7] in :0 \n at System.Net.WebCompletionSource1[T].WaitForCompletion () [0x000b1] in <cae6d5d40bdb48e7919fce5039029dd1>:0 \n at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x00118] in <cae6d5d40bdb48e7919fce5039029dd1>:0 \n at Duplicati.Library.Main.BackendManager.List () [0x00049] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x0000d] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00000] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x0010d] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x01031] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <6973ce2780de4b28aaa2c5ffc59993b1>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00008] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Controller+<>c__DisplayClass13_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x00035] in <c6c6871f516b48f59d88f9d731c3ea4d>:0 \n at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action1[T] method) [0x00271] in :0 \n at Duplicati.Library.Main.Controller.Backup (System.String inputsources, Duplicati.Library.Utility.IFilter filter) [0x00068] in :0 \n at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00307] in :0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“mscorlib”}
Running low on ideas… On Linux, old versions of mono sometimes do really unexpected remote issues. Would you happen to have a Windows system for reference? Alternatively, you can try to upgrade mono.
But the slash maybe-or-maybe-not red herring has returned. Not sure why it’s only sometimes escaped.
Drwtsn32: I didn’t intentionally escape the slash either, that got done internally by Duplicati. (I only saw the escape by looking at the underlying command-line format, under the 3 Dots at the top of the dialog.) But no need to follow-up on that now, as I’ve switched to the Master Key, without the slash.
Didn’t know about upgrade mono, will look into that.
EDIT: The mono upgrade didn’t help, sigh.
EDIT: Just noticed the mono repo only goes up to Fedora 29, I’m on Fedora 30, sigh.
Any time you use SSL or TLS you’re using certificates (maybe you’ve looked at those in a web browser?).
Transport Layer Security suggests that the “Authentication failed” is the inability of the system to verify the server certificate against the mono certificate store. You probably didn’t get far enough for B2 to reject you. Basically your computer trusts certain certificate authorities, and I suspect it doesn’t trust Backblaze B2’s.
FAQ: Security has a test you can try, and if it works you can then adapt it to something specific to B2 use:
csharp -e ‘new System.Net.WebClient ().DownloadString (“https://api.backblazeb2.com”)’
Above page has a list of suggestions. If you prefer, you could start testing at the Linux level without mono.
https://www.backblaze.com/b2/docs/b2_authorize_account.html has sample code for the curl command.
EDIT: and you can start by just pasting it in to see if it fails at the SSL/TLS level or with bad authentication.
Then maybe this issue can be worked around by using the --accept-any-ssl-certificate option? If that works, it could be locked down better with --accept-specified-ssl-hash. I use the latter with some family members’ computers that back up to my NAS at home (which has a self-signed cert).
1 Like
Thanks again. I understand about general security concepts, just not about how Duplicati handles things (esp. with B2 and other cloud stores), that’s why I mentioned “knowingly” using certs (I haven’t intentionally provided any to Duplicati). You mention “mono cert store,” but I know nothing about that. You mention browser, but as I mentioned I can use B2’s browser interface without problem. Your other suggestions look promising, as I know nothing about things like csharp (esp. see below for your csharp test, it also fails cert validation). Also, the suggestions about bypassing cert validation look promising, I’ll look into those too. Thanks again.
$- csharp
Mono C# Shell, type “help;” for help
Enter statements below.
csharp> new System.Net.WebClient().DownloadString(“https://api.backblazeb2.com/”)
System.Net.WebException: Error: TrustFailure (Authentication failed, see inner exception.) —> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. —> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
at /builddir/build/BUILD/mono-5.18.1.3/external/boringssl/ssl/handshake_client.c:1132
at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00054] in :0
at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x000c6] in :0
at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)
at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in :0
at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0012a] in :0
at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x000a4] in :0
— End of inner exception stack trace —
at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00336] in :0
at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x0018c] in :0
at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001f5] in :0
— End of inner exception stack trace —
at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x00275] in :0
at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x0015b] in :0
at System.Net.WebOperation.Run () [0x000b7] in :0
at System.Net.WebCompletionSource1[T].WaitForCompletion () [0x000b1] in <cae6d5d40bdb48e7919fce5039029dd1>:0 at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x00118] in :0
at System.Net.HttpWebRequest.GetResponse () [0x00019] in :0
at System.Net.WebClient.GetWebResponse (System.Net.WebRequest request) [0x00000] in :0
at System.Net.WebClient.DownloadBits (System.Net.WebRequest request, System.IO.Stream writeStream) [0x0011c] in :0
at System.Net.WebClient.DownloadDataInternal (System.Uri address, System.Net.WebRequest& request) [0x00069] in :0
at System.Net.WebClient.DownloadString (System.Uri address) [0x00011] in :0
at System.Net.WebClient.DownloadString (System.String address) [0x00008] in :0
at .Host (System.Object& $retval) [0x00006] in <7046bb68f2194f00bdb23c6ced21230e>:0
at Mono.CSharp.Evaluator.Evaluate (System.String input, System.Object& result, System.Boolean& result_set) [0x0003e] in :0
at Mono.CSharpShell.Evaluate (System.String input) [0x00000] in <445efc3aa14a4c44bc4de07e0f68cd4d>:0
csharp>
Success! Bypassing cert validation (–accept-any-ssl-certificate) works! Now I’ll go back and try other things, such as AppKey instead of MasterKey, etc., etc. Thanks again!
EDIT: I don’t really need to validate the B2 cert in any case. For, there’s enough security in other ways to guarantee integrity/confidentiality. Such as, pre-encryption with AES, and access to B2 storage via the B2 App/Master Key, and finally viewing B2 via browser to ensure the stuff really is getting there. Hackish, yes, but good enough to get me going for now …
EDIT: OK, so with mono is installed a “cert-sync” command, that I didn’t know about until now (never needed it, as I hadn’t tried cloud storage before). The magic incantation is: cert-sync /etc/pki/tls/certs/ca-bundle.crt. Thanks again!
EDIT: Regarding that business with “/” in keys (sometimes): It seems that has to do with Base64 hex encoding, see Base64 - Wikipedia.
It leaves authentication of https server to .NET Framework on Windows, or mono on other OSs, and these may delegate to something lower, such as OpenSSL on Linux, or Windows supplied facilities on Windows.
That’s because your web browser successfully authenticated that web server from its certificate. Duplicati isn’t your web browser. When using Duplicati, you browse to Duplicati’s server, then it connects to remote, which for Backblaze B2 is https://api.backblazeb2.com, but it’s an API, and won’t let my browser browse it.
is the kind of display I was talking about, often available from the browser padlock icon on an https:// site. Some browsers track trusted certificates themselves. Others use certificates that the OS has collected. mono seems to use its own certificate store, but has ways to get certificates into it from OS or browser.
I’m not sure exactly how much mono you installed when you did the upgrade. All of the directions except Red Hat related OSs warn you to make sure you really have the ca-certificates-mono package installed. Regardless, I’m glad cert-sync worked for you, because I think it’s best to verify certificates. Here’s why:
This is not entirely true, for example your B2 credentials could be read by an attacker doing a Man-in-the-middle attack, claiming to be api.backblazeb2.com, and without any certificate checking to spot an issue. Such tools (for example Fiddler) have legitimate uses for debugging, but you need to take steps to make them trusted by your client (unless of course there’s a way to tell your client to just trust anything it sees).
While I’d agree that the client-side encryption Duplicati does keeps an attacker from reading your source files, giving away your login information (as FTP also does unless one uses TLS in FTPS) is undesirable.
So I guess things are all set now, without needing any of the --accept* workarounds, and “/” hurts nothing.
If so, sounds good. 
I don’t think the man-in-the-middle attack works, because as I mentioned I can take a side-route via browser to view the B2 Bucket, and see that it’s getting populated/updated by Duplicati. The MITM can’t do that (because he doesn’t have the needed B2 creds). But, as mentioned, this is too hackish to depend upon.