Baaahhh... authentication failing after upgrading ubuntu

Possibly also look for this, from the issue. I don’t know if logging is always enabled:

On the destination machine, the following is logged for the sshd daemon:

userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms

That would imply that the problem was the known incompatibility with OpenSSH 8.8.

The OpenSSL 3 connection was tempting. It does break some other Duplicati things:

which is a genuine problem, but I’m not sure if it’s the explanation for your SSH pains.
Also recall that PuTTY broke somehow, so an sshd might be the common connection.

Duplicati runs on many systems. Not all have ssh. Those that do may have differences.
Duplicati is mostly portable C# code, and SSH.NET is too. This aids cross-platform use.

Portable code generally avoids native libraries needs, so OpenSSL uses would be rare.
An exception is the FasterHashing library from the above-cited problem. It goes fishing:

Wrapper library for using native hashing libraries on Windows, Linux and OSX