Anything interesting logged on the Linux FTP server side? I’m guessing it’s a TLS handshake issue, like maybe your server doesn’t support TLS 1.2 and Windows 11 refuses to use a lower version of TLS.
That’s the weird thing. It’s using TLS 1.3 and in the logs for the FTP server I can see the FTP user logging in.
It is something certificate based though. If I try and import a backup file into Duplicati and use a brand new folder I get an error about certificate being invalid, even if “accept-any-ssl-certificate” is added as an option.
This is also in the logs:
System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
at Duplicati.Server.WebServer.RESTMethods.RemoteOperation.CreateFolder(String uri, RequestInfo info)
at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
I’ve tried from a Windows 10 computer. Same network, same FTP server. The only difference is I have to accept all certificates again (as I tried recreating it on the server when I had the issues) and it uses a different user name, but that was successful. So it’s the same backend server, but doesn’t work on Win 11 using SSL in FTP in Duplicati, but it does in Windows 10 using SSL in FTP.
Can you tell me what Linux OS+version you are using, and what FTP daemon+version you are running? I’ll see if I can replicate the issue.
Thanks for the help.
It’s the latest version of Raspberry Pi OS running Pure FTP which looks to be 1.0.49-4.1.
In my testing so far, Windows 10 is fine connecting to it via Duplicati.
I can also transfer files from Windows 11 to the server using Filezilla without any problems.
Duplicati on Windows 11 does seem to be able to connect to the server and sort files, but I’m unable to do any further testing at the moment as I’ve had to rebuild the database for one of my backups as it seems to have go messed up with the testing.
This was part of the problem when I upgraded my systems last time. I had to check all systems and clients to support TLS 1.2 minimum. I also switched from FTPS to AFTP because I think there was a problem with FTPS and TLS 1.2 at least with Mono on Linux.
Thanks for this.
It does seem to be related to the encryption, but seems to also be local to Duplicati.
I can add, delete and reupload files to the same FTP sevrer, using the same user, using Windows 11 and Filezilla with no issues. But, if TLS is off, Duplicati does work.
Checking in Filezilla, it is using TLS 1.3.
By the way the AFTP reference meant the third-party one listed as FTP (Alternative) on the dropdown, useful when the standard Microsoft one has trouble. I don’t know if it will help, but might be worth a try.
Happy to give AFTP a try, but which option is best to force encryption? I’ve tried a few and can’t seem to get it to work but I don’t know what I’m doing.
After a few tests, Duplicati seems to just then get stuck saying “Testing Connection…”
The one that your server supports, but it helps if you know your server. Examine by netstat or telnet?
Methods of invoking security covers the difference between “implicit” and “explicit”, if that’s the issue.
You could possibly look at what your regular FTP used for a port. 21 implies explicit, and 990 implicit.
The thing is, the connection test works on the standard FTP settings with SSL ticked. But, when it comes to actually backing up, it fails with I assume the FTP commands.
My database is still rebuilding so I’m unable to test trying the AFTP with encryption as a proper test, but as soon as it has done I will give it a go.
In the limited testing I have done with the “Test connection” button for alternative FTP, I can’t get it to work.
Thanks for the info, I can set it to “Explicit” but the server throws an error of “Sorry, cleartext sessions and weak ciphers are not accepted on this server.#012Please reconnect using TLS security mechanisms.” Even if I set an option to use TLS 1.3 in Duplicati I get the same error.
I will do a bit more testing with standard FTP once my database has rebuilt.
But, I did find this which seems a similar issue:
I know the 451 error is usually server side, but I can confirm that the server works with Windows 10 and with Filezilla in Windows 11.
Also, Duplicati is able to successfully run a backup set if the source data hasn’t changed. It only fails when it is trying to add data to the repository.
I’ve managed to do some testing due.
If SSL is ticked for FTP, Duplicati is able to connect to the FTP server and is able to also put files there. It looks like the backup runs successfully all the way up to the end, in which it fails with the 451 error. But, the files are on the FTP server and there are no errors in the FTP Server logs.
I tried the Alternative FTP in Duplicati with explicit set, but this also fails. The server says “Sorry, cleartext sessions and weak ciphers are not accepted on this server” which I don’t understand as the server allows both encrypted and unencrypted traffic, but this fails. Duplicati says “System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. —> System.ComponentModel.Win32Exception: The function requested is not supported”.
Unchecking the “Use SSL” for FTP results in the backup working.
So, it seems there is some incompatibility between Windows 11 and secure FTP in Duplicati. These same backups were fine on Windows 10 with FTP with SSL, I’ve tested a different Windows 10 and Duplicati works with FTP with SSL and Windows 11 via Filezilla is also successful using FTP with SSL too.
And unfortunately, I can’t figure out a way to get it to work with the alternative FTP either.
I don’t have Windows 11, and have never put certificates on FTP. I can do deep debugging.
That would probably need you to take Wireshark traces on your system (because I cannot).
Did you try replacing the ftps in URL with aftp ?
This is the option which allows you to set the encryption mode…
Yet, there are problems with the aftp as well, as example it seems that some servers assume that the data connections also use the same key as the control connection. but Duplicati doesn’t seem to do that.
Operation List with file attempt 1 of 5 failed with message: Unable to build data connection: TLS session of data connection not resumed. => FluentFTP.FtpCommandException: Unable to build data connection: TLS session of data connection not resumed.
This could be related to the issue I saw earlier that one FTP connection is left open, during the data transfers. Afaik, that doesn’t make any sense. But without knowing how it’s designed to work, it’s hard to tell if there’s some purpose. But to me, it seems mostly and probably sloppy code, which gets the dirlist using connection A and then doesn’t use that connection for the worker threads transferring rest of files. Basically that one connection just idles. Maybe it’s a mistake, maybe it isn’t. Those are very common fails when quickly sticking stuff together without caring too much details. And at least in my case it does work without problems. The session just times out. Maybe that has something to do with the QUIT problem which I reported in another thread, or maybe it doesn’t who knows.
Sounds like the server and client cannot agree on what cipher to use with TLS. What is the cipher suite config of your FTP service?
I can set up a Windows 11 VM, but unfortunately I don’t have a spare Raspberry Pi to test with.
Cheers for the pointers. I’ll give it a try tomorrow and see if it works.
It’s using Pure FTP and it uses TLS1.3. The strange thing is that the server is set to fall back and use unencrypted (TLS setting 1) but for some reason it just doesn’t work with the encryption in this mode but I’ll give it another try tomorrow.
I can see in normal FTP mode with SSL set, it is able to communicate and even upload files. It seems something at the end of the backup is what causes the 451 error.
TLS 1.3 doesn’t tell us the cipher suite config (supported key exchange, authentication, encryption, and hashing algorithms).
I’ll check the documentation for PureFTP version you’re running to see if they mention it.
What exact option, set where? Also beware that some destination types tend to lose their advanced options on the Destination screen on an edit-again, and the workaround is to set on Options screen.
Might as well check a recent copy of the help text too. Sometimes it’s more current than the manual.
C:\ProgramData\Duplicati\duplicati-188.8.131.52_canary_2022-03-13>Duplicati.CommandLine.exe help aftp Alternative FTP (aftp): This backend can read and write data to an FTP based backend using an alternative FTP client. Allowed formats are "aftp://hostname/folder" or "aftp://username:password@hostname/folder" Supported options: --auth-password (Password): Supplies the password used to connect to the server The password used to connect to the server. This may also be supplied as the environment variable "AUTH_PASSWORD". --auth-username (String): Supplies the username used to connect to the server The username used to connect to the server. This may also be supplied as the environment variable "AUTH_USERNAME". --disable-upload-verify (Boolean): Disable upload verification To protect against network or server failures, every upload will be attempted to be verified. Use this option to disable this verification to make the upload faster but less reliable. --aftp-data-connection-type (Enumeration): Configure the FTP data connection type If this flag is set, the FTP data connection type will be changed to the selected option. * values: AutoPassive, PASV, PASVEX, EPSV, AutoActive, PORT, EPRT * default value: AutoPassive --aftp-encryption-mode (Enumeration): Configure the FTP encryption mode If this flag is set, the FTP encryption mode will be changed to the selected option. * values: None, Implicit, Explicit * default value: None --aftp-ssl-protocols (Flags): Configure the SSL policy to use when encryption is enabled This flag controls the SSL policy to use when encryption is enabled. * values: None, Ssl2, Ssl3, Tls, Default, Tls11, Tls12, Tls13 * default value: Default --aftp-upload-delay (Timespan): Add a delay after uploading a file Some FTP servers need a small delay before reporting the correct file size. The required delay depends on network topology. If you experience errors related to the upload size not matching, try adding a few seconds delay. * default value: 0s
So you probably wanted
aftp-ssl-protocols changed off of
Which key? I’m not super expert, but there appear to be private, public, and shared encryption keys.
The Illustrated TLS Connection is one complete but fairly readable (compared to the RFCs) writeup.
Securing FTP with TLS seems to be the RFC to read for protocol info, and it cites some earlier ones.
Raspberry Pi OS seems to be Debian-based, so maybe Pure-FTPd on a similar non-PI system will do.
TLS 1.3 is also quite different from 1.2, and for testing purposes it might be best to see if 1.2 can work.
Regarding cipher suite interaction, the best view is probably from Wireshark capturing port 21 traffic…
I think it’s ordinarily possible to probe the server (might need to aim at the implicit port for that here…).