Sftp Failed to connect: Permission denied (publickey)

Support OpenSSH 8.8 #4615 has this stack in Duplicati issues and links to a fixed SSH.NET issue.

Results of relevant openssl version and ssh -V might tell us if this version incompatibility exists.

If that is this problem, a new Duplicati would be needed to pick up the SSH.NET that has their fixes.

So a LinuxServer image, not a Duplicati. Chances are low that you’ll get one from the Duplicati team.
Occasionally the Duplicati developers will do private test builds, but they’re usually not Dockers at all.