You can use the ssh-keyfile advanced option to provide the path to your private key. Duplicati will then use this to authenticate with your server, which should have your public keys.
See [SOLVED] SFTP/SSH backups without password (pubkey only)? for related details.