FTP without SSL fails with 2.1.0.2_beta_2024-11-29

Support
1

Hi,
yesterday I upgraded from “duplicati-2.0.8.1_beta_2024-05-07-x64” (Windows 10) to 2.1.0.2_beta_2024-11-29. Since then FTP hangs and upload ends without success and with errors:


15. Dez. 2024 14:05: Die Operation Backup ist mit folgenden Fehler fehlgeschlagen: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)))
System.AggregateException: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)))

 ---> System.AggregateException: Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.))

 ---> FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

   --- End of inner exception stack trace ---

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

 ---> (Inner Exception #1) System.AggregateException: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)

 ---> FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

   --- End of inner exception stack trace ---<---



   --- End of inner exception stack trace ---

   at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter, CancellationToken token)

   at Duplicati.Library.Main.Controller.<>c__DisplayClass17_0.<Backup>b__0(BackupResults result)

   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
15. Dez. 2024 14:05: Fatal error
FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)
15. Dez. 2024 14:05: Backend event: Put - Failed: duplicati-bd509110a3d114827bbf215dc427a13e1.dblock.zip.aes (1,95 MB)
15. Dez. 2024 14:05: Operation Put with file duplicati-bd509110a3d114827bbf215dc427a13e1.dblock.zip.aes attempt 6 of 6 failed with message: Error while uploading the file to the server. See InnerException for more info.
FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)
15. Dez. 2024 14:05: Backend event: Put - Started: duplicati-bd509110a3d114827bbf215dc427a13e1.dblock.zip.aes (1,95 MB)

SFTP works, but is very slow on the small target machine. With the old version all worked fine all the time. I tried it with pureftp and proftp without success. The strange thing is: When setting up a backup task in Duplicati and using the FTP test connection in the settings page works fine. But when starting the backup, it hangs at “waiting for upload” and nothing is transmitted.

I also tried to downgrade back to Duplicati 2.0.8.1, but this is impossible, as Duplicati won’t startup at all. So I have to stay with 2.1.0.2 and using SFTP, which takes about 4x the time of FTP.

Any idea how to solve this with some settings? Thanks.

2

Welcome to the forum @Dumatolapi

Release: 2.1.0.2 (Beta) 2024-11-29 release note got a similar issue (fixed but not in Beta).
Is that environment similar to yours? If so, try the fix-by-options shown (and proven) there.

See Downgrade from 2.1.0.2 to 2.0.8.1 but I hope you don’t actually need to take this step.

3

Thank you. No, in my environment I don’t use encryption, only pure FTP (in my local network). If I understand the other thread correctly they have issues with TSL/encryption, which I don’t use.

Is there any other solution instead of downgrading?

Does anyone know the difference between FTP connection test in the task settings and the FTP upload during a task run? Why does testing work and running don’t? Thanks.

4

I suppose you could try this part of the fix to see if it helps. If not, wait for help from dev.

There are lots of other options you can try, but I can’t give further guesses on first to try.
FTP area required some revision, and I wasn’t who did that. I hope you can wait awhile.
Are you good with network tracing or server logs ? If you downgrade, info first may help.
If you can get a port 21 trace, be sure not to accidentally post your credentials to forum.

5

Thanks. I already tried --ftp-encryption-mode=Explicit and some others with no success. I did watch syslog, but I am not a Linux user. Network tracing I am not really into it.

Downgrade seems a little bit tricky, so I keep using SFTP until hopefully the FTP issue gets solved. Are the devs already aware of these kinds of FTP issues?

6

I can’t speak for what they know of that has not been posted.

On that note, maybe you can post your FTP configuration (with private info redacted).
If you use any Advanced options, this is especially important.

You already mentioned servers you test with. I think other issue used one of them too.

EDIT:

Works for me with a small no-SSL no-added-options backup to FileZilla Server on PC.
If yours is anything complicated, maybe you can find a smallest possible test that fails.

7

Do you meen the commandline?

"C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" backup "ftp://mylocalserver:21//mnt/daten/Temp/Testxxx?auth-username=user&auth-password=xxxxxxxxxxxxxx&ftp-absolute-path=true&ftp-data-connection-type=PASV&aftp-data-connection-type=PASV&aftp-encryption-mode=None" "C:\Users\user\Desktop\files\\" --backup-name=Test --dbpath="C:\Users\user\AppData\Local\Duplicati\IEHLVYSBSL.sqlite" --backup-id=DB-5 --encryption-module=aes --compression-module=zip --dblock-size=50mb --passphrase="xxxxxxxxxxxx" --keep-time=1D --disable-module=console-password-input
8

That’s one good way, although an Advanced options screenshot might have been OK.
You seem to have three. I can get a problem with them, and stop it by unchecking this:

image
image1118×190 14.2 KB

but I don’t know if that suits your use. Anyway, now some progress in understanding it.

9

I did some other testing with no options and I did it with FTP and FTP alternative. It works with “test connection”, but not with a backup run after saving the task. This is really strange.

10

These are less separate than they were before, seemingly due to a Microsoft deprecation.

Removing the legacy FTP Backend #5589

That’s usually just a file list as sanity check. I think aftp had a better one, but it looks like the rearranged code arranged ftp to have it. Here’s server log of upload from a Test connection:

<Date/Time> Info [Type] Message
<12/15/2024 11:24:46 AM> FTP Session 5 127.0.0.1 duplicati [Command] STOR /test%202/duplicati-access-privileges-test.tmp

Another thing that worries me is that it URL encoded a space. I’m not sure that should happen.
I’m inclined to let the people who are more familiar with ftp changes continue the investigation.

11

Yes, this was another issue for me after upgrade. I had a space in folder name and the backup was broke. But after resolving this by changing it to “-” instead of space I found out the other issue, we are discussing here. If I only kept the old version…

12

FTP (Alternative) path fails with spaces and other special characters #4587
was a previous now-fixed issue. I haven’t tested much this time, but I do get

C:\FTP\test%202

where the space in test 2 became a %20.

13

We are reading the forum daily :slight_smile:

As @ts678 mentions, the FtpWebRequest has been deprecated for a while, so we had to change it at some point.

I think the FTP issue is fixed in the latest canary build, which should be easier to try than downgrading:

I would think it works with:

--ftp-encryption-mode=None

(The aftp-* settings are only applied if the url starts with aftp://).

Then I would think it is related to the active/passive setting perhaps?

That should not happen. It was introduced with the fix to support absolute paths, and should be fixed in code.

14

Thank you. I already tried 2.1.0.102_canary_2024-12-12 without success. I did it again now and tried all --ftp-data-connection-type= parameters with FTP (not alternative) - none did work.

--ftp-encryption-mode=None also does not make any difference. I still can test the connection with “ok” and the real backup hangs with “waiting for upload” and does not upload anything.

As mentioned before: I used 2.0.8.1_beta_2024-05-07-x64 for months with “FTP alternative” (without any parameters) and it worked fine all the time. Since my upgrade to 2.1.0.2_beta_2024-11-29 it stopped working with FTP (and alternative) at all, I can only use SFTP.

15

FYI: I have made a PR to correct that.

16

Thanks! Do you have any idea about the other problem?
The reporter was having trouble that I wasn’t able to get.

17

As for the other issue, I did search and found what seems to indicate a FluentFTP issue closely related to setting the encryption mode.

If anything I would suggest @Dumatolapi to experiment with Implicit if he has not done so already, to rule out that and if we could know more about the ftp server being used, maybe we can try to replicate the scenario.

–ftp-encryption-mode=Implicit

The modes available are:

namespace FluentFTP
{
  /// <summary>Defines the type of encryption to use</summary>
  public enum FtpEncryptionMode
  {
    /// <summary>Plain text.</summary>
    None,
    /// <summary>
    /// FTPS encryption is used from the start of the connection, port 990.
    /// </summary>
    Implicit,
    /// <summary>
    /// Connection starts in plain text and FTPS encryption is enabled
    /// with the AUTH command immediately after the server greeting.
    /// </summary>
    Explicit,
    /// <summary>
    /// FTPS encryption is used if supported by the server, otherwise it falls back to plaintext FTP communication.
    /// </summary>
    Auto,
  }
}

Also one thing to be noted is that FTP and FTP Alternate are now exactly the same backend using FluentFTP library.

18

I am not sure if I tested this with “Implicit”, but only with “none” and I would like to test it now, but at the moment I am not able to test any more, because my SoC server died last night - perhaps it has something to do with the higher load because of SFTP. It does not even start up, have to check out what’s going on.

Now I am searching for a replacement SoC…

1 Like
19

Uiuiui, I opened my SBC and found a very bad taste, which came from the power supply. As I feared the SBC drew more power than before because of encryption via SFTP.

Now I tested “Implicit”, but then I get even an error in the settings on “Test connection”: Failed to connect: Cannot determine the frame size or a corrupted frame was received.

On “explicit” I get: Failed to connect: AUTH TLS command failed.

So no solution here.

20

This seems right (to me at least) if the goal is to not encrypt, as topic subject suggests.

Mine works fine from Windows 10 to pure-ftpd on Linux. That was one that you tested?

Because I never set up the FTP server with a certificate, I expect such use to not work.

Implicit fails with “Cannot determine the frame size or a corrupted frame was received.”
Interestingly, “Test connection” works, and it would interesting to see how that happens.
Same with Explicit Test, however the Backup run gets an “AUTH TLS command failed.”

I’m doing the above with FTP. Now let’s try FTP (Alternative). I seem to have to ask for

image
image1108×176 14.1 KB

but 2.0.8.1 has a similar issue. Anyway, Test connection now works, so go for backup.

2024-12-18 11:55:09 -05 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option --ftp-encryption-mode is not supported and will be ignored

because I still had that from when it was ftp. Delete that, save, and the backup runs fine.

I’m working with new jobs, and Settings has no Default options that might be interfering.

As a side complaint, the GUI has hung twice while doing this. One was at start, other was:

image

There’s no run-script-after to explain this, but browser (Edge) refresh clears up leftover.

In terms of this topic, I can’t get FTP hung when it shouldn’t. Any network boxes in its path? Firewalls and routers can sometimes need different setup. My test uses no extra equipment.

image
image1124×163 9.21 KB

is one way to see details for hangs. On Linux server, maybe tcpdump -i any -A port 21

EDIT:

The FTP (Alternative) errors for aftp-encryption-mode Implicit and Explicit are just like FTP.

Test connection working or not seems to vary. It acts like Advanced options are taken from Destination screen but not Options screen. If so, that’s probably inconsistent with other use.