Windows 11 Secure FTP Problem?

ts678 · March 30, 2022, 5:19pm

What exact option, set where? Also beware that some destination types tend to lose their advanced options on the Destination screen on an edit-again, and the workaround is to set on Options screen.

Might as well check a recent copy of the help text too. Sometimes it’s more current than the manual.

C:\ProgramData\Duplicati\duplicati-2.0.6.101_canary_2022-03-13>Duplicati.CommandLine.exe help aftp
Alternative FTP (aftp):
 This backend can read and write data to an FTP based backend using an
 alternative FTP client. Allowed formats are "aftp://hostname/folder" or
 "aftp://username:password@hostname/folder"
 Supported options:
  --auth-password (Password): Supplies the password used to connect to the
    server
    The password used to connect to the server. This may also be supplied as
    the environment variable "AUTH_PASSWORD".
  --auth-username (String): Supplies the username used to connect to the
    server
    The username used to connect to the server. This may also be supplied as
    the environment variable "AUTH_USERNAME".
  --disable-upload-verify (Boolean): Disable upload verification
    To protect against network or server failures, every upload will be
    attempted to be verified. Use this option to disable this verification to
    make the upload faster but less reliable.
  --aftp-data-connection-type (Enumeration): Configure the FTP data connection
    type
    If this flag is set, the FTP data connection type will be changed to the
    selected option.
    * values: AutoPassive, PASV, PASVEX, EPSV, AutoActive, PORT, EPRT
    * default value: AutoPassive
  --aftp-encryption-mode (Enumeration): Configure the FTP encryption mode
    If this flag is set, the FTP encryption mode will be changed to the
    selected option.
    * values: None, Implicit, Explicit
    * default value: None
  --aftp-ssl-protocols (Flags): Configure the SSL policy to use when
    encryption is enabled
    This flag controls the SSL policy to use when encryption is enabled.
    * values: None, Ssl2, Ssl3, Tls, Default, Tls11, Tls12, Tls13
    * default value: Default
  --aftp-upload-delay (Timespan): Add a delay after uploading a file
    Some FTP servers need a small delay before reporting the correct file
    size. The required delay depends on network topology. If you experience
    errors related to the upload size not matching, try adding a few seconds
    delay.
    * default value: 0s

So you probably wanted aftp-ssl-protocols changed off of Default, not allowed-ssl-versions.

Which key? I’m not super expert, but there appear to be private, public, and shared encryption keys.
The Illustrated TLS Connection is one complete but fairly readable (compared to the RFCs) writeup.

Securing FTP with TLS seems to be the RFC to read for protocol info, and it cites some earlier ones.

Raspberry Pi OS seems to be Debian-based, so maybe Pure-FTPd on a similar non-PI system will do.

TLS 1.3 is also quite different from 1.2, and for testing purposes it might be best to see if 1.2 can work.

Regarding cipher suite interaction, the best view is probably from Wireshark capturing port 21 traffic…
I think it’s ordinarily possible to probe the server (might need to aim at the implicit port for that here…).

bkev · March 31, 2022, 7:57am

Cheers for the pointers. I will try to do some more testing over the next couple of days. But, Duplicati does work with TLS1.3 in Windows 10. I had no issues there, so it seems to be when I upgraded to Windows 11 the errors occurred.

Also worth noting that using the FTP with SSL in Duplicati in Windows 11, Duplicati is able to connect to the server and upload files. It’s right at the end of the backup it tries to do something which fails the job. Using Win 11 with the SSL selected I can watch the folder on the server and see the files appearing, so it seems there isn’t an issue with connectivity, just something at the end of the job that throws an error 451.

The same FTP user worked in Win 10 and works in FileZilla on Win 11 but I might not be driving the same FTP command as Duplicati in FileZilla (I’ve tested creating folders, uploading files, downloading files and deleting files and it all works fine).

bkev · March 31, 2022, 2:16pm

Break through!

The issue is something to do with Duplicati on Windows 11 using TLS 1.3.

The setup: the backup is using standard FTP with SSL ticked.

I’ve changed a setting on the server:
/etc/ssl/openssl.cnf
To
MinProtocol = TLSv1.2

Restarted Pure FTP.

At this point, Duplicati still fails.

I can see on the server that Duplicati is trying to contact the server using TLS 1.3. Again, is actually worth noting that the computer and server are talking as files are put on the server, but at the end of the backup it fails.

So, if I add the option in the backup profile of allowed-ssl-versions and select TLS12 and TLS13 again it fails and the server shows the connection using TLS 1.3.

Unselecting TLS13 in allowed-ssl-versions, this then forces Duplicati to contact the server on TLS 1.2, which I can also confirm on the server.

This backup is then successful.

So, it seems that Duplicati, Windows 11 and TLS1.3 don’t play nice!

ts678 · March 31, 2022, 3:06pm

About → Show log → Live → Retry will probably let you see what that is. Try clicking on errored lines.
A backup is usually a lot of uploads (put), but at the end there could be delete, get, and list also done.

bkev · March 31, 2022, 3:07pm

Looking through my logs, it might be Duplicati and TLS1.3 in general.

It might be that the move to Windows 11 tries to use TLS1.3 by default now, but looking back Duplicati was connecting via TLS1.2, so I’m not sure if TLS1.3 actually works even on Windows 10!

bkev · March 31, 2022, 3:12pm

Operation Put with file duplicati-be…c69e.dblock.zip.aes attempt 6 of 6 failed with message: The remote server returned an error: (451) Local error in processing.

ts678 · March 31, 2022, 3:17pm

A dblock is typically a somewhat large (default 50 MB but the last one might be smaller) file with blocks of data from the source file. Typically a backup puts up a series of full ones intermixed with dindex files which index a dblock and so are much smaller. What sort of files were getting through prior to this upload failing?

bkev · March 31, 2022, 3:30pm

I can see the dblocks on the FTP server but also failures for the same dblock in the live log. Now, depending on how accurate the file size is on the files system of the FTP server and the log…

Backend event: Put - Failed: duplicati-b1…dfa1.dblock.zip.aes

But I can see this file on the FTP server.

Operation Put with file duplicati-b1…edfa1.dblock.zip.aes attempt 6 of 6 failed with message: The remote server returned an error: (451) Local error in processing.

{“ClassName”:“System.Net.WebException”,“Message”:“The remote server returned an error: (451) Local error in processing.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at System.Net.FtpWebRequest.DataStreamClosed(CloseExState closeState)\r\n at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)\r\n at System.Net.FtpDataStream.Dispose(Boolean disposing)\r\n at System.IO.Stream.Close()\r\n at Duplicati.Library.Backend.FTP.d__19.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__24.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<>c__DisplayClass20_0.<b__0>d.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__21.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__21.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__20.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__18.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<b__13_0>d.MoveNext()",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nDataStreamClosed\nSystem, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a34e089\nSystem.Net.FtpWebRequest\nVoid DataStreamClosed(System.Net.CloseExState)”,“HResult”:-2146233079,“Source”:“System”,“WatsonBuckets”:null}

ts678 · March 31, 2022, 4:35pm

That’s weird. If you want to cut down on delay and noise, change Options number-of-retries=0.
You can also get rough file sizes (why is size being mentioned?) in live log or log file. Example:

2022-03-31 09:52:57 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b1ebe7e335641477f99ad8fd9b0e8e339.dblock.zip.aes (49.95 MB)

and you’ll notice I got seconds on that. That’s because it’s log-file=<path> with log-file-log-level.
You can probably get a fairly precise idea of whether the whole file got up before odd 451 error.

though, was asking about earlier files in the same backup. Was all going well until things broke?

bkev · March 31, 2022, 6:43pm

Ok, so after some more testing, when TLS 1.3 is set, Duplicati is only able to upload duplicati-b…92fd8881.dblock.zip.aes files and NOT .dindex.zip.aes files.

So, when TLS 1.3 is set, every single file starts duplicati-b and ends with dblock.zip.aes and there are no other files at all.

Without changing anything on the server, changing to TLS1.2, I get all the files.

ts678 · April 1, 2022, 12:01am

Well, I’m stumped. The files are uploaded the same way. The only difference is the file length.
For test (not for practical backups) you can reduce the Remote volume size as low as 1 MB.

It looks like you’re back on the Microsoft client. Maybe FTP (Alternative) could be tried more?

bkev · April 1, 2022, 8:51am

1MB doesn’t work it gets the same error.

I’ve not been able to get the alternative client to work at all.

Setting it to use TLS 1.3 and Explicit in Duplicati makes it fail to connect, with the server complaining about “Sorry, cleartext sessions and weak ciphers are not accepted on this server” which doesn’t make sense as the server is able to accept TLS1.2, TLS1.3 and I’ve also allow no TLS, so that’s just weird.