SFTP backup no longer working on Asustor NAS after patch

Support
1

Me an my friend both have an Asustor NAS. We have both installed Duplicati from the NAS AppStore and created jobs that backup the data via SFTP to each others NAS. Of course we have also configured everything required around that like DNS entries, firewall ports etc. and it has been running for many months now without any issues.

A few days ago we got the notification on our NAS that the new patch 4.2.5.RN33 is available. We both installed it and after the restart the backups were no longer working. The error message in Duplicati is “An established connection was aborted by the server.” Nothing else in our setup has changed, smoke tests show no issues with anything around it (IP is correct, ports are open etc.) and we both have the exact same problem after the exact same action (patch installation) so I clearly assume that this patch is the cause of the problem.

I checked the changelog but couldn’t really see anything suspicious (Release Notes - ASUSTOR NAS). Or could it be related to the OpenSSH fix?

I feel the new patch is changing (fixing) the behavior of the SFTP server in such a way that Duplicati can no longer communicate with it. So from my understabding a fix could be required either on the NAS or on Duplicati side.

Anybody else is observing this problem? How to analyze / drill down / fix this?

Disclaimer: I have created a similar issue in the Asustor forum: ASUSTOR Community Forum • View topic - SFTP no longer working after patch 4.2.5.RN33

1 Like
2

Hello

did you try to set allowed-ssl-versions to Tls 1.2 ?

3

Just tried it now but same result…

4

Are you able to access the files with another SFTP client or directly with SSH?

The TLS issue is with FTPS, so that doesn’t apply here.

5

Allowed-ssl-versions is an advanced option for Sftp.

6

In addition to trying other SFTP clients or even SSH, have you tried setting up a new Duplicati job? Possibly something changed that somehow invalidated the ssh-fingerprint that got stored long ago.
You don’t need a complete job, just a Destination screen where you can click Test connection.

Well, if you want to drill rather deep, you can see how far it gets before its connection gets closed.
9 - Cryptography Basics - SSH Protocol Explained shows Wireshark lines to expect. Ignore detail. Approximately identifying the problem spot is a nice start (just look for a few lines) on the analysis.

SSH is not SSL. Duplicati does use SSL/TLS in many places, so config is potentially a global one.

is the idea I mean. Technically, people on the web think SecurityProtocol is per application domain.
How to use SSL3 instead of TLS in a particular HttpWebRequest?

7

Yes,thanks for the correction, I replied too fast.

Actually, resetting the ssh fingerprint can be done using the very same job. Just use test and Duplicati will ask to update it.

1 Like
8

Yes, I have tried creating a completely new job in Duplicati. I also tried most other advanced settings such as accept any fingerprint or accept any certificate but it keeps failing with the same error message.

I have tried connecting with FileZilla and that works without any issues. To me that seems to indicate a problem with Duplicati, probably some new security algorithm that Duplicati doesn’t (properly) support.

In order for you to test it yourselves I have temporarily opened access to my SFTP server. Try to connect to 184.82.24.166 at port 12345 and you should see that a) it works with FileZilla and probably other clients but b) fails with Duplicati. (Use any username / password - if you get an authentication error it works, otherwise you won’t even get there.)

9

From Test connection click on Destination screen:

image

Testing from Windows 10.

10

I cannot confirm that. I get:
image
My configuration looks like this:

image
image667Ă—646 19.6 KB

11

image
image848Ă—800 36.6 KB

The ssh-fingerprint was of course not invented by me, but I accepted it. Maybe you didn’t get so far?

12

What Duplicati version are you on? I found one from 2018 (2.0.4.5). Is there a current one (2.0.7.1)?
You can check your version in the GUI on the About screen.

13

I’m on 2.0.7.1_beta_2023-05-25. That’s what it automatically downloads on Asustor and I believe this is the newest version.

No, I don’t get to accept the fingerprint, it fails before that.

14

Similar issues in SSH.NET (the library Duplicati uses):

This one seems to be caused by version 2023.0.0, which is strange because it looks like we are still using 2020.0.2. Maybe your app store updated the DLL? That might explain why it only fails in your case.

This one fails after a server update on version 2020.2, but if your server uses an incompatible algorithm, it should not work for anyone with Duplicati. This one would be fixed by version 2023.

If the fingerprint is an indication, the algorithm is ecdsa-sha2-nistp256, which is supported on the older version.

15

what is even stranger is that I can repro the problem with Mono, but upgrading SSH.NET to 2023.0.0 fixes it.

16

Good that you can reproduce it, at least we don’t have to check the DLL version. So it probably is an incompatible key algorithm. Can you see something in the ConnectionInfo, as suggested in the second issue?

@srudin Do you have access to the server logs to see what the server-side error is?

17

I have full access to the NAS but no idea where to find this information on Asustor. I cannot see anything in the system logs but I can search for other log files if you have me a path or some other hints to look for.

18

@srudin

Did you change something on your NAS ? Since the previous access, I can’t repro either the first error I saw (it fails but differently), or the success I had with recent SSH.NET version.

19

I have no Asustor, and barely run sshd, but if Asustor uses systemd, try systemctl status sshd.
When testing with Duplicati 2.0.4.5 (too old, but I’m not sure if it’s relevant), I can get this sshd line:

Unable to negotiate with 192.168.86.81 port 52343: no matching host key type found. Their offer: ssh-rsa,ssh-dss [preauth]

Above test was on Manjaro, which (being kind of bleeding edge) has the new openssh Asustor took.
Sometimes Linux systems (which I barely run) log to files in /var/log. You can ls -lrt it.

Do you have a Windows system around to test Duplicati? If not, maybe a more complete Linux OS?
Something with a GUI is nice if you want to examine packets, although Asustor might have tcpdump packet capture, so packets could be moved around if necessary. The failure to yours from 2.0.4.5 is:

image

I still can’t find that Asustor App Central has 2.0.7.1, so possibly it started on something old, then Duplicati autoupdater moved it onto latest version in Beta channel. What’s About → System info?
BaseVersionName is the original install. ServerVersionName is what’s running after autoupdate.

EDIT 1:

My Windows 2.0.7.2 Canary (identical to 2.0.7.1 Beta) to my Manjaro OpenSSH_9.5p1 looks OK, although I’m stopping at the Trust host certificate step after clicking the Test connection.

EDIT 2:

I went looking for why Windows Duplicati was working for me to Asustor unlike other tester results.
I’m still not an SSH protocol expert, and Duplicati does not build its own, just like Asustor does not.
Interoperability failures can be inability to negotiate common algorithms. Did Asustor remove one?

server_host_key_algorithms:

Asustor server  rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256
Manjaro server  rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519

Linux   client  ssh-ed25519,ssh-rsa,ssh-dss
Windows client  ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-rsa,ssh-dss

With Asustor lacking ssh-ed25519 (compared to Manjaro), there’s no Linux to Asustor match there.
Windows Duplicati can match Asustor on ecdsa-sha2-nistp256. I wonder what prior Asustor had?

Getting Asustor logs would also be helpful, as sshd is better at computing its complaint than I am…
Packet capture would be another way to tackle it. I had trouble installing Wireshark on Linux, so ran:

tcpdump -i any -w asustor.sftp.pcap port 12345

and did my viewing on my existing Windows Wireshark. Double-click Server: Key Exchange Init

1 Like
20

That explains why version 2023 works, because it adds support for rsa-sha2. I don’t know why the linux client does not allow the NIST elliptic curves when Windows does.
Version 2020.0.2 should support them, maybe it is a mono thing:

SSH.NET supports the following host key algorithms:

  • ssh-ed25519
  • ecdsa-sha2-nistp256
  • ecdsa-sha2-nistp384
  • ecdsa-sha2-nistp521
  • ssh-rsa
  • ssh-dss

@srudin If you are able to change the /etc/ssh/sshd_config file, try to add ssh-ed25519,ssh-rsa:

HostKeyAlgorithms
Specifies the host key signature algorithms that the server
offers. The default for this option is:

            ssh-ed25519-cert-v01@openssh.com,
            ecdsa-sha2-nistp256-cert-v01@openssh.com,
            ecdsa-sha2-nistp384-cert-v01@openssh.com,
            ecdsa-sha2-nistp521-cert-v01@openssh.com,
            sk-ssh-ed25519-cert-v01@openssh.com,
            sk-ecdsa-sha2-nistp256-cert-v01@openssh.com,
            rsa-sha2-512-cert-v01@openssh.com,
            rsa-sha2-256-cert-v01@openssh.com,
            ssh-ed25519,
            ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,
            sk-ssh-ed25519@openssh.com,
            sk-ecdsa-sha2-nistp256@openssh.com,
            rsa-sha2-512,rsa-sha2-256

         The list of available signature algorithms may also be
         obtained using "ssh -Q HostKeyAlgorithms".

I believe the defaults were changed to remove those old algorithms.

1 Like