GDrive backend `TrustFailure` (even with `accept-any-ssl-certificate`)

[fph]

Greetings. I am encountering trouble with my remote backups, using Duplicati version 2.0.2.1 under Ubuntu 17.10. It seems like the problems started a few days ago after I updated from 17.04. Performing a backup using an existing configuration with Google Drive backend fails at the beginning of the procedure with the following error

Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service:
Error: TrustFailure (The authentication or decryption has failed.).
If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=googledrive 
---> System.Net.WebException: Error: TrustFailure (The authentication or decryption has failed.) 
---> System.IO.IOException: The authentication or decryption has failed. 
---> System.IO.IOException: The authentication or decryption has failed.
---> Mono.Security.Protocol.Tls.TlsException: Invalid certificate received from server. 
Error code: 0xffffffff800b010a 
at Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (System.IAsyncResult asyncResult) [0x00040] in <1d0bb82c94e7435eb09324cf5ef20e36>:100:
at Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (System.IAsyncResult ar, System.Boolean ignoreEmpty) [0x00000] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
at Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (System.IAsyncResult result) [0x00071] in <1d0bb82c94e7435eb09324cf5ef20e36>:0

The rest of the log entry is:

--- End of inner exception stack trace --- at Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (System.IAsyncResult result) [0x0003b] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 at Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (System.IAsyncResult asyncResult) [0x0000c] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 --- End of inner exception stack trace --- at Mono.Security.Protocol.Tls.SslStreamBase.EndRead (System.IAsyncResult asyncResult) [0x00057] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 at Mono.Net.Security.Private.LegacySslStream.EndAuthenticateAsClient (System.IAsyncResult asyncResult) [0x00011] in <bd46d4d4f7964dfa9beea098499ab597>:0 at Mono.Net.Security.Private.LegacySslStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x0000e] in <bd46d4d4f7964dfa9beea098499ab597>:0 at Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) [0x00044] in <bd46d4d4f7964dfa9beea098499ab597>:0 --- End of inner exception stack trace --- at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <1cb5198b00f34ae59d97ee7fe7a3a16c>:0 at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse () [0x00044] in <1cb5198b00f34ae59d97ee7fe7a3a16c>:0 at Duplicati.Library.JSONWebHelper.GetResponse (Duplicati.Library.Utility.AsyncHttpRequest req, System.Object requestdata) [0x000b4] in <138bf26c6c1d46ad83e0ec8ca32c67c5>:0 --- End of inner exception stack trace --- at Duplicati.Library.Main.BackendManager.List () [0x00038] in <118ad25945a24a3991f7b65e7a45ea1e>:0 at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x0000d] in <118ad25945a24a3991f7b65e7a45ea1e>:0 at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00000] in <118ad25945a24a3991f7b65e7a45ea1e>:0 at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x00066] in <118ad25945a24a3991f7b65e7a45ea1e>:0 

Even generating a new OAuth token does not change anything.
Running sudo /usr/local/bin/cert-sync /etc/ssl/certs/ca-certificates.crt, as suggested by some other sources, does not fix the problem. Actually even if I set accept-any-ssl-certificate in the backup options (in step 5 of the config wizard), the backup starts and seems to run correctly almost until the end (after “Completing backup …”, "Waiting for upload … ", "Verifying backend data … "), when I get basically the same error:

Fatal error
Duplicati.Library.Interface.UserInformationException: 
Failed to authorize using the OAuth service: 
Error: TrustFailure (The authentication or decryption has failed.). 
If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=googledrive 
---> System.Net.WebException: Error: TrustFailure (The authentication or decryption has failed.) 
---> System.IO.IOException: The authentication or decryption has failed. 
---> System.IO.IOException: The authentication or decryption has failed. ---> Mono.Security.Protocol.Tls.TlsException: Invalid certificate received from server. Error code: 0xffffffff800b010a
  at Mono.Security.Protocol.Tls.RecordProtocol.EndReceiveRecord (System.IAsyncResult asyncResult) [0x00040] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
  at Mono.Security.Protocol.Tls.SslClientStream.SafeEndReceiveRecord (System.IAsyncResult ar, System.Boolean ignoreEmpty) [0x00000] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
  at Mono.Security.Protocol.Tls.SslClientStream.NegotiateAsyncWorker (System.IAsyncResult result) [0x00071] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 

The rest of the log entry is:

   --- End of inner exception stack trace ---
  at Mono.Security.Protocol.Tls.SslClientStream.EndNegotiateHandshake (System.IAsyncResult result) [0x0003b] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
  at Mono.Security.Protocol.Tls.SslStreamBase.AsyncHandshakeCallback (System.IAsyncResult asyncResult) [0x0000c] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
   --- End of inner exception stack trace ---
  at Mono.Security.Protocol.Tls.SslStreamBase.EndRead (System.IAsyncResult asyncResult) [0x00057] in <1d0bb82c94e7435eb09324cf5ef20e36>:0 
  at Mono.Net.Security.Private.LegacySslStream.EndAuthenticateAsClient (System.IAsyncResult asyncResult) [0x00011] in <bd46d4d4f7964dfa9beea098499ab597>:0 
  at Mono.Net.Security.Private.LegacySslStream.AuthenticateAsClient (System.String targetHost, System.Security.Cryptography.X509Certificates.X509CertificateCollection clientCertificates, System.Security.Authentication.SslProtocols enabledSslProtocols, System.Boolean checkCertificateRevocation) [0x0000e] in <bd46d4d4f7964dfa9beea098499ab597>:0 
  at Mono.Net.Security.MonoTlsStream.CreateStream (System.Byte[] buffer) [0x00044] in <bd46d4d4f7964dfa9beea098499ab597>:0 
   --- End of inner exception stack trace ---
  at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <1cb5198b00f34ae59d97ee7fe7a3a16c>:0 
  at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse () [0x00044] in <1cb5198b00f34ae59d97ee7fe7a3a16c>:0 
  at Duplicati.Library.JSONWebHelper.GetResponse (Duplicati.Library.Utility.AsyncHttpRequest req, System.Object requestdata) [0x000b4] in <138bf26c6c1d46ad83e0ec8ca32c67c5>:0 
   --- End of inner exception stack trace ---
  at Duplicati.Library.Main.BackendManager.List () [0x00038] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x0000d] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00000] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.PostBackupVerification () [0x00058] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x007f6] in <118ad25945a24a3991f7b65e7a45ea1e>:0 

The output of Tlstest is:

mono /usr/lib/duplicati/utility-scripts/TlsTest.exe --stream https://www.google.com

https://www.google.com
[Subject]
  CN=www.google.com, O=Google Inc, L=Mountain View, S=California, C=US

[Issuer]
  CN=Google Internet Authority G2, O=Google Inc, C=US

[Not Before]
  11/29/2017 10:47:51 AM

[Not After]
  2/21/2018 10:37:00 AM

[Thumbprint]
  63738898F2769D2FEC4B3A2D8B9C59F273452943


	Valid From:  11/29/2017 10:47:51 AM
	Valid Until: 2/21/2018 10:37:00 AM

Error #-2146762486: CERT_E_CHAINING 0x800B010A

Note that Tlstest connects without errors to other domains, though:

mono /usr/lib/duplicati/utility-scripts/TlsTest.exe --stream https://www.github.com

https://www.github.com

Any ideas on how I can solve this issue? My best attempt as an explanation currently is that (1) I have a cert store issue, and it is not clear to me how it should be fixed, and at the same time (2) there is a bug in duplicati that causes the final verification not to use option accept-any-ssl-certificate when it should.

[JonMikelV]

It sounds like you’re not the only one:

Failed to backup to Google Drive after couple of months

Since two days I get the following error message when backing up to Google Drive: As suggested by the message, I tried renewing the authid token for Google Drive, but that does not help either. I run Duplicati 2 version 2017-08-01 - 2.0.2.1_beta_2017-08-01. Does anyone know what is going on and how I can fix this problem? Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: GetResponse timed out. If the problem persists, try generating a new aut…

One time this error was reported (though not with Google Drive) here’s what @kenkendk had to say:

Amazon Cloud Drive - Web UI Hangs on 'Test connection'

It sounds like there is some connection issue between your machine and the OAuth server. I assume you can open https://duplicati-oauth-handler.appspot.com in a browser from the same machine. There is this issue, which indicates that sometimes the connections fail, but I am not sure it is related to your issue, as you get no reply (timeout):

[fph]

Thanks! I managed to get a successful backup following the suggestion by ElRico on github:

For future reference, I’ve kinda figured this out. Using openssl s_client -connect duplicati-oauth-handler.appspot.com:443, I saw that the certificate chain included a retired Equifax certificate as root. Since it is not being distributed anymore, I got hold of a copy from this page and added it to the machine trust store via certmgr -add -c -m Trust Equifax.crt. Why this is still required I can’t say.

So it seems there is indeed a Mono trust store issue (my browser can connect to duplicati-oauth-handler.appspot.com:443 just fine). In addition, should I report upstream the bug that the final verification does not use accept-any-ssl-certificate?

[JonMikelV]

@fph

In addition, should I report upstream the bug that the final verification does not use accept-any-ssl-certificate?

I’m not very familiar with the certificate side of things so without being able to verify anything myself, I’d say yes. If you can log in on GitHub that would be great (I did a quick search for accept-any-ssl-certificate and didn’t find any posts I consider duplicates of your final verification issue).

[Robert_Kirchner]

Thanks for the reply. Unfortunately, I got ‘sync: error opening ‘/etc/ssl/certs-ca-certificateds.crt’: No such file or directory’.

I was advised to remove the mono package because its connection to Microsoft makes it a security risk. So mono is uninstalled on my machine. Is there some other way to get those security certificates?

[fph]

Check the spelling of certificates.

I don’t think you can run Duplicati at all without Mono.

[Robert_Kirchner]

Federico Poloni suggested that my removal of Mono might be the problem. I fixed the spelling problem, reinstalled Mono, reinstalled Duplicati (just in case), synced the ssl certificates list again, ran a test on the connection, and got the same error message. I proceeded to run the backup again, with ‘accept any ssl certificate’ option, and got the same oauth failure. I also disabled my firewall, in case that was the problem, but it didn’t help.

[jan]

I am having exactly the same errors. Last backup I managed to Drive was 20 December. I use Ubuntu 17.10 and Duplicati - 2.0.2.1_beta_2017-08-01

Tried everything Robert_Kirchner has done but nothing helps.