$ ls -lu --full-time /etc/ca-certificates/update.d
total 8
-rwxr-xr-x 1 root root 2683 2021-10-03 21:36:15.866202325 -0400 jks-keystore
-rwxr-xr-x 1 root root 303 2021-10-03 21:36:18.466202405 -0400 mono-keystore
Ubuntu (not surprisingly) has it too, and mono-keystore was read 8 milliseconds before cert-sync.
The idea seems like a safer one than relying on mono updates, but I wonder if they ever disagree.
Thanks for pointing it out.
I think this might be from mono ca-certificates package, at least on Debian and derivatives:
Package: ca-certificates-mono (6.8.0.105+dfsg-3.2)
This package uses the hooks of the ca-certificates package to update the Mono keystore.
- dep: ca-certificates (>= 20121114)
$ dpkg-query --search /etc/ca-certificates/update.d/mono-keystore
ca-certificates-mono: /etc/ca-certificates/update.d/mono-keystore
$ dpkg-query --listfiles ca-certificates-mono
/.
/etc
/etc/ca-certificates
/etc/ca-certificates/update.d
/etc/ca-certificates/update.d/mono-keystore
/usr
/usr/bin
/usr/bin/cert-sync
/usr/lib
/usr/lib/mono
/usr/lib/mono/4.5
/usr/lib/mono/4.5/cert-sync.exe
/usr/share
/usr/share/doc
/usr/share/doc/ca-certificates-mono
/usr/share/doc/ca-certificates-mono/changelog.Debian.gz
/usr/share/doc/ca-certificates-mono/copyright
Interesting description here thanks @ts678 !
I use Duplicati on Docker v2.0.6.3, any idea if there is a simple way to workaround?
I donât run Docker. As a side note, Duplicati Monitoring is a completely separate project from Duplicati.
https://www.duplicati-monitoring.com/ says that, and gives some other contact paths, including email.
You have important data kept on your host (not in Docker, which is discarded during upgrades), right?
If so, you might see if there are enough tools in the Docker to try one of the documented workarounds.
The simplest workaround (which still violates the philosophy of don-t-change-the-Docker) would be for revised ca-certificates package to be released, then maybe it can be installed. Unfortunately, itâs stuck:
https://tracker.debian.org/pkg/ca-certificates
Things may change, but at the moment thereâs a grave bug (seemingly in scripting, not in certificates):
Debian Bug report logs - #996005 â ca-certificates: fails upgrading when no new certs selected
Although Duplicati staff is busy (overly busyâŚ) just with Duplicati, I think @drwtsn32 runs Duplicati by Duplicati Docker image (there are others â whatâs yours?), so might be able to comment on your issue.
I would strongly prefer that Duplicati Monitoring serve a Linux Duplicati compatible certificate to its users.
This is the same request I make for any service that breaks Duplicati. Most storage has been OK so far.
Backblaze was covered earlier, and the big ones often donât use Letâs Encrypt. Google is their own CAâŚ
The solution for the official Duplicati Docker image is the same as posted above for Debian. You want to remove this file from within the container:
/usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
Great its working now on my Docker instances! Thank you guys!
The only one still not working is the Duplicati plugin on my TrueNAS. Since this is FreeBSD based, the way certificates are managed slightly differs and I did not succeed.
Answering to myself: actually the solution is quite simple:
I started receiving this same error today when trying to connect to Backblaze B2 as the backup target.
The instructions by @hairlesshobo worked to fix the issue.
Thanks, @hairlesshobo!
Hi, I also get a cert error when trying to run my desktop backups.
System.Net.WebException: Error: TrustFailure (Authentication failed, see inner exception.) â> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. â> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
Thing is I am on Manjaro and since itâs not Debian based I canât apply the above workarounds. I also couldnât find anything tangible to do manually online, so I am at a bit of a loss here, and I donât want to try taking risky initiatives since this is my daily driver machine.
Does anyone happen to know what I could do/read?
I have a fully updated system. Backups were fine yesterday.
Thereâs a link in Duplicati issue Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED #4075 giving Arch advice.
Oh, I checked there but it wasnât mentioned yet 
Thank you @ts678 for the help, it worked.
For anyone else interested, the commands for Arch mentioned in the link are those below, that should be done as root:
# blacklist the expired certificate
cp /etc/ssl/certs/DST_Root_CA_X3.pem /etc/ca-certificates/trust-source/blocklist/
# update the system wide trust store
update-ca-trust
# and finally sync the system trust store to mono
cert-sync /etc/ssl/certs/ca-certificates.crt
Edit: You may need to run the last command twice.
Iâve been having an issue for the last week, but Iâm not sure if it is the same one from this thread or not.
Environment:
2 Fedora systems started failing Tuesday, November 30, the other Fedora system says it succeeded on the 30th, but then has failed since December 1. The Windows 10
system is not reporting any issues.
The error reported is:
System.AggregateException: One or more errors occurred. (Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
at /builddir/build/BUILD/mono-6.12.0.122/external/boringssl/ssl/handshake_client.c:1132 (Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
[repeated multiples times - I can provide the full trace if desired]
I have attempted to blacklist the DST certificate - no change, so either it wasnât the issue, or the attempt failed. Also, note that backups have been working through November so if the issue was the certificate expired in October, would this even be applicable?
I have also tried playing with forcing SSL3, TLS1.1, etc. (not saying I tried all of the them) - no help.
I tried âupdate-crypto-policies --set LEGACYâ followed by a reboot - no help.
Any suggestions for troubleshooting this?
Does the above RHEL/CentOS article help any? Is that how you blacklisted the certificate?
Do you see web page data or an error if you run this test, based on mono-project.com test:
csharp -e ânew System.Net.WebClient ().DownloadString (âhttps://api.backblazeb2.comâ)â
You could also ask what Backblaze support has to say. Above-named site (one of many you may trip on) has a Letâs Encrypt certificate with trouble-for-mono default chain, valid from 30 Nov 2021 02:22:14 UTC.
I did a âupdate-ca-trust extractâ, extracted the âDST Root CA X3â section to a file in /etc/pki/ca-trust/source/blacklist, and then ran update-ca-trust to rebuild the cert db (or whatever it does). The output certainly appears to be honoring the blacklist - I get multiple lines saying âp11-kit: overriding trust for anchor in blacklist: DST.crtâ. I also ran several cert-sync commands against files in the /etc/pki directory. To be extra safe, I then rebooted.
No web page data, no error. It just returns to the command prompt. Also, firefox has no trouble with the backblaze url. I will take a look at the mono-project link provided to see if it provides any further insight.
So you are saying that instead of a systemic issue, this would be specific to mono? If so I need to back-out the blacklist change. The Nov 30 date would be when 2 out of 3 systems started failing. Unless something was cached, Iâm not sure why the 3rd took an extra 24 hours to fail.
Thanks very much for the suggestions!
The âcsharp -e ânew System.Net.WebClient ().DownloadString (âhttps://www.nuget.orgâ)ââ test from the mono-project.com test link provides the expected html.
As stated previously, the " csharp -e ânew System.Net.WebClient ().DownloadString (âhttps://api.backblazeb2.comâ)â" test results in no output and no error.
I believe I have found a solution/workaround! Testing now, and will post details once I have confirmed.
Environment:
Issue:
Starting December 1, 2021, Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
Solution/Workaround:
And then as root:
Letâs Encrypt Chain of Trust has that link described as the ISRG Root X1 self-signed version.
Certificate Compatibility under Platforms that trust ISRG Root X1 doesnât list Fedora, but might not be an exhaustive list. Regardless, thanks for finding and documenting the addition that solved the issue for you.
I wish this was included in the docker image so that i didnt have to do it manually!
I tried this solution in Centos 8 and no luck, the CERTIFICATE_VERIFY_FAILED error persists. Duplicati canât get past the âListing remote filesâ step.
Anyone find a solution for Centos 8 or its successors like Rocky or Alma?