FTP without SSL fails with 2.1.0.2_beta_2024-11-29

I did a test with a new job without any options with FTP (alternative) and no encryption - still the same issue. There is no firewall or something else.

With the older version 2.0.8.1 FTP (alternative) did work for months without any issue. As FTP service was implemented “new” in the current version, I don’t think I can do anything except for waiting that someone will fix the issue in the “new” Duplicati FTP service.

PS: FTP with e.g. WinSCP using FTP without encryption also works fine with my SBC (pure ftp). The only issue I have with the current Duplicati version.

I don’t think anyone can fix it until they can see it, so the devs might still need your assistance.

At least you are not seeing the Please call Connect() ... error message with that version, right?

I still think you need --ftp-encryption-mode=None (as that is what you want).
For reference:

  • Implicit means “connect with TLS, same as HTTPS is doing, all data is encrypted”. This is IMO the only secure way to do FTPS, but is rarely used because the FTP client needs to know in advance to use it.
  • Explicit means: connect in plain-text, then send an AUTH command to start TLS.
  • Auto means, try explicit mode, but ignore AUTH failure and use None. An attacker can fake the AUTH response and trick the caller into sending the username and password in plain-text.
  • None means, just send username and password in plain-text. Anyone listening on the network can read the credentials.

Since the LIST call works, but the actual transfers fail, I am guessing this is due to the data channel using a wrong configuration. You can try the setting:

--ftp-data-connection-type=AutoActive

Since the default is passive, this will switch to active mode. Some of the other options are listed with explanations here:

I am sorry, but neither --ftp-encryption-mode=None nor --ftp-data-connection-type=AutoActive or both does help:

Warnings 1

    2024-12-19 08:23:06 +01 - [Warning-Duplicati.Library.Main.Operation.Backup.UploadSyntheticFilelist-MissingTemporaryFilelist]: Expected there to be a temporary fileset for synthetic filelist (1, duplicati-20241218T211158Z.dlist.zip), but none was found?

Errors 2

    2024-12-19 08:23:56 +01 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error

    FtpException: Error while uploading the file to the server. See InnerException for more info.
    2024-12-19 08:23:56 +01 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: Die Operation Backup ist mit folgenden Fehler fehlgeschlagen: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)))

    AggregateException: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)))
System.AggregateException: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)))

 ---> System.AggregateException: Error while uploading the file to the server. See InnerException for more info. (Error while uploading the file to the server. See InnerException for more info.) (One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.))

 ---> FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.Client.BaseClient.BaseFtpClient.HasFeature(FtpCapability cap)

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

   --- End of inner exception stack trace ---

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

   at CoCoL.ChannelExtensions.WaitForTask(Task task)

 ---> (Inner Exception #1) System.AggregateException: One or more errors occurred. (Error while uploading the file to the server. See InnerException for more info.)

 ---> FluentFTP.Exceptions.FtpException: Error while uploading the file to the server. See InnerException for more info.

 ---> FluentFTP.Exceptions.FtpException: Please call Connect() before trying to read the Capabilities!

   at FluentFTP.Client.BaseClient.BaseFtpClient.get_Capabilities()

   at FluentFTP.Client.BaseClient.BaseFtpClient.HasFeature(FtpCapability cap)

   at FluentFTP.AsyncFtpClient.FileExists(String path, CancellationToken token)

   at FluentFTP.AsyncFtpClient.UploadFileInternalAsync(Stream fileData, String localPath, String remotePath, Boolean createRemoteDir, FtpRemoteExists existsMode, Boolean fileExists, Boolean fileExistsKnown, IProgress`1 progress, CancellationToken token, FtpProgress metaProgress)

   --- End of inner exception stack trace ---

   at CoCoL.AutomationExtensions.RunTask[T](T channels, Func`2 method, Boolean catchRetiredExceptions)

   at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend(BackupResults result, IWriteChannel`1 uploadtarget, Task uploader)

   at Duplicati.Library.Main.Operation.BackupHandler.RunAsync(String[] sources, IFilter filter, CancellationToken token)

   --- End of inner exception stack trace ---<---



   --- End of inner exception stack trace ---

   at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter, CancellationToken token)

   at Duplicati.Library.Main.Controller.<>c__DisplayClass17_0.<Backup>b__0(BackupResults result)

   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

Is this with 2.1.0.102 ? I would expect the “Please call Connect()” issue to be fixed in 2.1.0.102.

Yes, it’s 2.1.0.102_canary_2024-12-12.

Thanks for confirming. I will try to set up a replication using PureFTPd in Docker.

I have now tested various configurations with PureFTPd, and I cannot replicate the errors you describe. It works as expected with various settings for --ftp-data-connection-type (did not try the Active modes as that requires open firewall on the client).

Even using various faulty setups, I cannot reproduce neither the case where the Test works but not upload/download. If I block the ports it will give a timeout, but I never get the “half-works” state. I have not seen the “Please call Connect()” error message at any time during testing.

The weird part is that for the FTP backend, there is actually a full set of tests where it lists, uploads, downloads and deletes inside the test function. So all operations should be working if the test completes as you see.

One thing that is different between the previous implementation (FtpWebRequest) and FluentFTP is that the FtpWebRequest would create a brand new connection for each request, where FluentFTP uses the more efficient method of keeping the connection open and sends multiple requests. Not sure why that would make a difference though, but just stating it in case it helps.

Looking at the symptoms, I would guess that something is closing the connection once a large(r) file is being transferred. But it is super strange that both ProFTPd and PureFTPd would behave the same.

Did you try to transfer a larger file with FileZilla to check that the connection truly works?
I tried with 200MiB files with no issues in my setup.

Thank you! Yes, I use the SBC (Banana Pi Pro) as a small backup system. As mentioned it worked for months with Duplicati with normal FTP (alternative) without connection encryption. I also use Personal Backup (https://personal-backup.rathlev-home.de) as another backup system on the same SBC with very large files (since several years now). All fine (except for the burnt power supply :wink:). So at least pureftp is running normal.

I am not a Linux expert, so I have only limited knowledge of setting up things on the Armbian System on my SBC, but I get around with SSH, bash etc.

If there is anything I could help with to find out the cause, I will try.

My way of getting split is this:

Yes, the destination page’s Test button does not look at any other options than what is on the page. I guess there could be an option in the “Advanced options” on the last step that breaks something? Or maybe in general options?

Let’s ask @Dumatolapi which screen the options were on. Also check the Setting screen.

As a note, we were pushing Screen 5 for awhile, as Screen 3 dropped options during Edit.

I thought Default options in Settings doesn’t offer backend specific options via dropdown.
Probably one could type them in, so I guess we should ask about that since neither of us is reproducing the problem. What’s different, I wonder? FWIW, I tested a very small backup…

No, there aren’t any settings, I made a new task with no options at all. Only a simple test task for FTP.

But there are news: As I mentioned I am not a Linux nerd, but thankfully with the help of ChatGPT I found out, that my SBC runs proftpd instead of pure-ftpd. So any changes to pure-ftp.conf did not make any sense.

Is there anything I should check in the proftpd.conf? I did not change anything, so all settings are default. But if I change the port e.g., this works. So perhaps there is a setting for proftpd to get it working with Duplicati?

PassivePorts 30000 35000

# Deaktiviere FEAT-Anfragen, um Probleme zu vermeiden
<IfModule mod_feat.c>
    FeatureEnabled off
</IfModule>


#
# /etc/proftpd/proftpd.conf -- This is a basic ProFTPD configuration file.
# To really apply changes, reload proftpd after modifications, if
# it runs in daemon mode. It is not required in inetd/xinetd mode.
#

# Includes DSO modules
Include /etc/proftpd/modules.conf

# Set off to disable IPv6 support which is annoying on IPv4 only boxes.
UseIPv6 on
# If set on you can experience a longer connection delay in many cases.
<IfModule mod_ident.c>
  IdentLookups off
</IfModule>

ServerName "Debian"
# Set to inetd only if you would run proftpd by inetd/xinetd/socket.
# Read README.Debian for more information on proper configuration.
ServerType standalone
DeferWelcome off

# Disable MultilineRFC2228 per https://github.com/proftpd/proftpd/issues/1085
? # MultilineRFC2228on
DefaultServer on
ShowSymlinks on

TimeoutNoTransfer 600
TimeoutStalled 600
TimeoutIdle 1200

DisplayLogin welcome.msg
DisplayChdir .message true
ListOptions "-l"

DenyFilter \*.*/

# Use this to jail all users in their homes
# DefaultRoot~

# Users require a valid shell listed in /etc/shells to login.
# Use this directive to release that constrain.
# RequireValidShelloff

# Port 21 is the standard FTP port.
Port 21

# In some cases you have to specify passive ports range to by-pass
# firewall limitations. Ephemeral ports can be used for that, but
# feel free to use a more narrow range.
# PassivePorts 49152 65534

# If your host was NATted, this option is useful in order to
# allow passive tranfers to work. You have to use your public
# address and opening the passive ports used on your firewall as well.
# MasqueradeAddress 1.2.3.4

# This is useful for masquerading address with dynamic IPs:
# refresh any configured MasqueradeAddress directives every 8 hours
<IfModule mod_dynmasq.c>
# DynMasqRefresh 28800
</IfModule>

# To prevent DoS attacks, set the maximum number of child processes
# to 30.  If you need to allow more than 30 concurrent connections
# at once, simply increase this value.  Note that this ONLY works
# in standalone mode, in inetd mode you should use an inetd server
# that allows you to limit maximum number of processes per service
# (such as xinetd)
MaxInstances 30

# Set the user and group that the server normally runs at.
User proftpd
Group nogroup

# Umask 022 is a good standard umask to prevent new files and dirs
# (second parm) from being group and world writable.
Umask 022 022
# Normally, we want files to be overwriteable.
AllowOverwrite on

# Uncomment this if you are using NIS or LDAP via NSS to retrieve passwords:
# PersistentPasswd off

# This is required to use both PAM-based authentication and local passwords
# AuthOrder mod_auth_pam.c* mod_auth_unix.c

# Be warned: use of this directive impacts CPU average load!
# Uncomment this if you like to see progress and transfer rate with ftpwho
# in downloads. That is not needed for uploads rates.
#
# UseSendFile off

TransferLog /var/log/proftpd/xferlog
SystemLog /var/log/proftpd/proftpd.log

# Logging onto /var/log/lastlog is enabled but set to off by default
#UseLastlog on

# In order to keep log file dates consistent after chroot, use timezone info
# from /etc/localtime.  If this is not set, and proftpd is configured to
# chroot (e.g. DefaultRoot or <Anonymous>), it will use the non-daylight
# savings timezone regardless of whether DST is in effect.
SetEnv TZ :/etc/localtime

<IfModule mod_quotatab.c>
QuotaEngine off
</IfModule>

<IfModule mod_ratio.c>
Ratios off
</IfModule>


# Delay engine reduces impact of the so-called Timing Attack described in
# http://www.securityfocus.com/bid/11430/discuss
# It is on by default.
<IfModule mod_delay.c>
DelayEngine on
</IfModule>

<IfModule mod_ctrls.c>
ControlsEngine off
ControlsMaxClients 35
ControlsLog /var/log/proftpd/controls.log
ControlsInterval 5
ControlsSocket /var/run/proftpd/proftpd.sock
</IfModule>

<IfModule mod_ctrls_admin.c>
AdminControlsEngine off
</IfModule>

#
# Alternative authentication frameworks
#
#Include /etc/proftpd/ldap.conf
#Include /etc/proftpd/sql.conf

#
# This is used for FTPS connections
#
#Include /etc/proftpd/tls.conf

#
# This is used for SFTP connections
#
#Include /etc/proftpd/sftp.conf

#
# This is used for other add-on modules
#
#Include /etc/proftpd/dnsbl.conf
#Include /etc/proftpd/geoip.conf
#Include /etc/proftpd/snmp.conf

#
# Useful to keep VirtualHost/VirtualRoot directives separated
#
#Include /etc/proftpd/virtuals.conf

# A basic anonymous configuration, no upload directories.

# <Anonymous ~ftp>
#   User ftp
#   Group nogroup
#   # We want clients to be able to login with "anonymous" as well as "ftp"
#   UserAlias anonymous ftp
#   # Cosmetic changes, all files belongs to ftp user
#   DirFakeUser on ftp
#   DirFakeGroup on ftp
#
#   RequireValidShell off
#
#   # Limit the maximum number of anonymous logins
#   MaxClients 10
#
#   # We want 'welcome.msg' displayed at login, and '.message' displayed
#   # in each newly chdired directory.
#   DisplayLogin welcome.msg
#   DisplayChdir .message
#
#   # Limit WRITE everywhere in the anonymous chroot
#   <Directory *>
#     <Limit WRITE>
#       DenyAll
#     </Limit>
#   </Directory>
#
#   # Uncomment this if you're brave.
#   # <Directory incoming>
#   #   # Umask 022 is a good standard umask to prevent new files and dirs
#   #   # (second parm) from being group and world writable.
#   #   Umask022  022
#   #   <Limit READ WRITE>
#   #     DenyAll
#   #     </Limit>
#   #       <Limit STOR>
#   #         AllowAll
#   #     </Limit>
#   # </Directory>
#
# </Anonymous>

# Include other custom configuration files
# !! Please note, that this statement will read /all/ file from this subdir,
# i.e. backup files created by your editor, too !!!
# Eventually create file patterns like this: /etc/proftpd/conf.d/*.conf
#
Include /etc/proftpd/conf.d/

As mentioned FTP with other apps is working flawlessly.

Arh, so no need chasing PureFTPd.

We have tested quite a lot with ProFTPd and not seen this issue either.

This could help, but you can get even more detailed logs if you add:

TraceLog /var/log/proftpd/trace.log
Trace ALL

And then restart ProFTPd:

sudo systemctl restart proftpd

If you then run:

tail -f /var/log/proftpd/proftpd.log

You should be able to see commands as they are sent. If you try to run the backup again, we should be able to figure out what exactly goes wrong. If it is not obvious what the problem is, you can perhaps compare the logs with what you see in another session, using a client that works.

Thanks, got it working (instead of ALL I used DEFAULT:10). Here is the log.
trace.zip (12.5 KB)

Thanks for that :star_struck:

I have looked through the log and it seems to just have entries for:

  • login
  • feature setup
  • file list
  • quit

And no errors.
Only odd thing I could find was that after the initial LIST there is a 50 second pause, and then a QUIT command. From the logs, the server has 600s delay before closing the connection, so presumably the client has sent the command.

Was this during an attempted backup?
This is super odd, because there is no other connection attempt apparently made.
If this is correct, then I am leaning towards the issue that @ts678 has uncovered in the thread about too many clients, where it creates two connections: one for listing and one for transfers.

This could explain it if there is something preventing the second connection from the same source IP.

Could you try to start two instances of the FTP client that is working, and see if both instances can connect?

50s seems to be the time from starting upload and waiting to finish, this is shown in the green bar in Duplicati.

Yes, it was a normal run of my simple test backup.

Yes, I can open WinSCP with FTP two times (even 3 I tested) and upload is working.

At least now I can see why there are two different instances of the FTP client.

At some point we introduced parallel uploads to better saturate upload speed.
Since each backend instance is not required to be thread-safe, the implementation creates a new instance for each uploader thread (defined by --asynchronous-concurrent-upload-limit).

But, parallel uploads were never it the original design, which is focused on a queue of backend transactions handled by BackendManager.

To avoid changing the code that uses BackendManager, the parallel upload code creates new instances. This means there is one connection for BackendManager which does the listing and verify, but each upload handler uses a different connection.

For any tests that I have been able to run, this is fine, albeit strange, so I am thinking that there could be something on your client operating system that limits it to 1 connection per ip:port:process. So the second connection stalls, because there is already an active connection.

I cannot find any settings or software for Windows that might do this, but there is mention of a “20 connection limit” on Windows 10 Home.

Any chance you have some kind of limiter on Windows? It has to be here because the process id is not known outside the machine, so that could only do a total connection limit.

No, I don’t have any kind of this or such settings, at least I don’t know of such settings in Windows.

And as I tested today I can easily make more than one FTP connection from my PC to the SBC. Also remember: I used Duplicati for months in the older version without any issue. This does not seem to be the cause for me or is this FTP feature a new one?

Yes, but that would be different processes right?

The previous version of the FTP client was a bit inefficient, as it would do the entire auth/feature handshake before starting the transaction, and then exiting.

While it worked on your system, it is not how FTP is intended to work, as you should have a single control connection and then start transfers from there.

I was looking at fixing the issue so there is only a single control connection, but unfortunately it requires rewriting the way the transfers are handled, so it is not a quick-fix by any means. I have registered an issue for fixing the backend transfer logic, which would also (presumably) fix this issue as well.