Release: 2.0.9.103 (Canary) 2024-08-15

2.0.9.103_canary_2024-08-15

This release is a canary release intended to be used for testing in preparation of a later stable release.

** Unlike regular canary builds, this one has a major change in the build system, so it now runs on .NET8. **

For that reason, the updater in previous canary builds does not detect this update yet, but this can be activated at a later time.

** Important changes from last Beta **

  • Updated to .NET8 with OS specific builds
  • Using Kestrel as the API/UI server
  • Mandatory password and new authentication scheme
  • Settings database version updated to v7

Please see list of known issues related to .NET8/Kestrel upgrade:

Detailed list of changes:

  • Multiple updates for styling and visual consistency, thanks @luixxiul
  • Extensive work on making all documentation strings follow a consistent logic, thanks @luixxiul
  • Fixed support for Websocket over https, thanks @Riches
  • Fixed an issue with --webservice-allowed-hostnames being renamed
  • Fixed support for captcha on systems without the Arial font
  • Using different default filenames for logging with TrayIcon and Server
  • Fixed issue with livelog not showing contents
  • Fixed an issue where a login issue would not show an option to log in
  • Fixed showing correct port number in log output
  • Fixed not showing scheduler state and next backup
  • Fixed an issue where a crash would show the wrong stack trace
  • Fixed an issue with generating massive number of inotify watchers
  • Re-enabled the button to log out
  • Added option to disable the visual captcha
  • Added support for providing server commandline arguments via environment variables
  • Fixed log error message related to update download url
  • Added Telegram reporting module

This was a good load of fixes. Thanks! Anyone else running it? Some complex cases still fail.
This test is on Windows 10 system called HP4, testing from a VM on it (test external access).

I had thought that the accidental webservice-allowedhostnames was supposed to work as webservice-allowed-hostnames did. It seems to in some ways. In other ways, it does not, however if synonymns are hard, maybe just going back to support only original name is fine?

I’m generally testing using URL 192.168.86.81:8200 to see if IP address URLs get access.
Settings screen help (but not the shorter CLI help) is quite clear that this isn’t name-checked:

To prevent various DNS based attacks, Duplicati limits the allowed hostnames to the ones listed here. Direct IP access and localhost is always allowed. Multiple hostnames can be supplied with a semicolon separator. If any of the allowed hostnames is an asterisk (*), all hostnames are allowed and this feature is disabled. If the field is empty, only IP address and localhost access is allowed.

--webservice-interface=any --webservice-allowed-hostnames="*"
Allows password login

--webservice-interface=any --webservice-allowedhostnames="*"
Remote browser gets
Bad Request - Invalid Hostname
HTTP Error 400. The request hostname is invalid.

--webservice-interface=any --webservice-allowedhostnames="allowedhostnames"
Remote browser gets
Bad Request - Invalid Hostname
HTTP Error 400. The request hostname is invalid.

--webservice-interface=any --webservice-allowed-hostnames="hp4"
Crashes with terminal output
Unexpected error: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 400 (Bad Request).)
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable`1 timeout)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternal[T](String method, String endpoint, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

Wireshark shows the port 8200 traffic as
POST /api/v1/auth/signin HTTP/1.1
Host: localhost:8200
User-Agent: Duplicati-TrayIcon-Monitor/2.0.9.103
Content-Type: application/json; charset=utf-8
Content-Length: 265

{"SigninToken":"REDACTED"}
HTTP/1.1 400 Bad Request
Content-Length: 334
Content-Type: text/html
Date: Fri, 16 Aug 2024 18:13:05 GMT
Server: Kestrel

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></ HEAD >
<BODY><h2>Bad Request - Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p>
</BODY></HTML>

--webservice-interface=any --webservice-allowed-hostnames="allowed-hostnames"
Crashes with terminal output
Unexpected error: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 400 (Bad Request).)
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 400 (Bad Request).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable`1 timeout)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternal[T](String method, String endpoint, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

--webservice-interface=invalid
Crashes with terminal output
Unhandled exception. Duplicati.Library.Interface.UserInformationException: Server crashed on startup
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at System.ThrowHelper.Throw(String paramName)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.Server.Program.get_StatusEventNotifyer()
   at Duplicati.Server.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper.<>c__DisplayClass5_0.<.ctor>b__0(Object dummy_arg)
   --- End of inner exception stack trace ---
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper..ctor(String[] args)
   at Duplicati.GUI.TrayIcon.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.Net8.Program.Main(String[] args)

--no-hosted-server seems to break it. It’s not even sending anything on localhost 8200.

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Duplicati.GUI.TrayIcon.exe --no-hosted-server

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Unexpected error: System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at System.ThrowHelper.Throw(String paramName)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainSignInToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

I’m on Windows, wondering how to get TrayIcon connected to Duplicati as Windows service.

I have the same problem, --no-hosted-server does nothing and neither does the TrayIcon appear.
Backups though with this version work.

As a Windows service? To even browse in, I had to add a password. Trying to config it off got:

A serious error occurred in Duplicati: System.Exception: Disabling password protection is not supported
   at Duplicati.Server.Database.ServerSettings.SetWebserverPassword(String password)
   at Duplicati.Server.Program.AdjustApplicationSettings(Dictionary`2 commandlineOptions)
   at Duplicati.Server.Program.Main(String[] _args)
Unhandled exception. System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at System.ThrowHelper.Throw(String paramName)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.Server.Program.get_StatusEventNotifyer()
   at Duplicati.Server.Program.Main(String[] _args)
   at Duplicati.Server.Net8.Program.Main(String[] args)

so I configured a (maybe) temporary password, e.g. --webservice-password="temporary"

TrayIcon getting to a password protected service was pretty roundabout in old design as well.

Another note on the service was that watching its processes (Process Explorer is nice) showed that the Server processes were coming and going, which is sometimes a symptom of it deciding to exit. Adding a --log-file just got me an empty log file, so I opened a cmd.exe as SYSTEM and found

C:\>C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui\Duplicati.Server.exe
Server is now running on port 8200
Initial signin url: http://localhost:8200/signin.html?token=REDACTED
Unhandled exception. System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'IServiceProvider'.
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.Server.Program.get_StatusEventNotifyer()
   at Duplicati.Server.Program.SignalNewEvent(Object sender, EventArgs e)
   at Duplicati.Server.Program.<>c.<SetWorkerThread>b__62_4()
   at WebserverCore.Services.SchedulerService.<>c__DisplayClass6_0.<SubScribeToNewSchedule>b__0(Object _, EventArgs _)
   at Duplicati.Server.Scheduler.Runner()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)

I will try to test some more. The idea with alias was that both options could be in use, and that is why both are supposed to work.

I have not tested with IP addresses, that might be what is causing some problems. Previously, the hostname check was implemented in the Duplicati code. The new setup is making the hostname check using the built-in ASP.NET hostname check, and that might also reject IP calls. I have registered an issue for this.

If I understand you setup correctly, this is the exact issue that the hostname check is preventing. You have set the allowed hostname to be hp4 but you are sending in localhost:8200.

This one has changed a bit due to the stricter password storage. In previous versions, the TrayIcon would read the WebUI password from the database if none was provided. This allowed a tray+server on the same filesystem to work seamlessly. With the updated password storage, the password can no longer be read, only validated.

For that reason you need to also pass in the password for the trayicon when using --no-hosted-server. The error message should make this super clear. I have added an issue for improving this situation.

An aside is that we may be able to go back to having the password “readable” using this PR.

It should work with:

Duplicati.TrayIcon.exe --no-hosted-server --hosturl=http://localhost:8200 --webservice-password=<password>

That suggests that the server has already crashed for another reason, and some event is trying to signal the crashed server. Do you happen to see anything else?

Alternatively, is this reproducible by just running:

Duplicati.WindowsService.exe INSTALL

Or do I need to change something else?

It’s possible, but the accidental one was only the documented one for less than two weeks.
Canary is expected to be a slightly bumpy ride, and 2.0.9.102 was an exceptionally bumpy.

This is sort of my feeling from seeing 2.0.9.102 behaviors, but there was a lot of other noise.

Although you wouldn’t know it from its help text, this scenario used to work in 2.0.8.1. Tested

I thought the “seam” was that the TrayIcon may not have permission to a SYSTEM database. Whatever the situation of current Beta, the need now is the next design, so let me continue…

Regardless (and per prior posts) the inability to disable server password seems to require the manual password set, so I don’t fully understand steps for a Windows service. Need a guide?

After fixing the command name and using my password, still exactly nothing on the network.
Previous network watch was Wireshark on port 8200. Current is broader in Process Monitor.

2.0.8.1 with only a --no-hosted-server looks like this (as opposed to a blank GUI screen):

7:16:46.7390882 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Reconnect	0:0:0:0:0:0:0:1:65350 -> 0:0:0:0:0:0:0:1:8200	SUCCESS	0.0000000	7:16:46.7390882 AM	Length: 0, seqnum: 0, connid: 0
7:16:47.2476681 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Reconnect	0:0:0:0:0:0:0:1:65350 -> 0:0:0:0:0:0:0:1:8200	SUCCESS	0.0000000	7:16:47.2476681 AM	Length: 0, seqnum: 0, connid: 0
7:16:47.7598165 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Reconnect	0:0:0:0:0:0:0:1:65350 -> 0:0:0:0:0:0:0:1:8200	SUCCESS	0.0000000	7:16:47.7598165 AM	Length: 0, seqnum: 0, connid: 0
7:16:48.2607797 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Reconnect	0:0:0:0:0:0:0:1:65350 -> 0:0:0:0:0:0:0:1:8200	SUCCESS	0.0000000	7:16:48.2607797 AM	Length: 0, seqnum: 0, connid: 0
7:16:48.2608759 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Disconnect	0:0:0:0:0:0:0:1:65350 -> 0:0:0:0:0:0:0:1:8200	SUCCESS	0.0000000	7:16:48.2608759 AM	Length: 0, seqnum: 0, connid: 0
7:16:48.2754193 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Connect	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.2754193 AM	Length: 0, mss: 65495, sackopt: 1, tsopt: 0, wsopt: 1, rcvwin: 2619800, rcvwinscale: 8
7:16:48.2797245 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.2797245 AM	Length: 229, startime: 13602514, endtime: 13602514, seqnum: 0, connid: 0
7:16:48.4774014 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.4774014 AM	Length: 128, seqnum: 0, connid: 0
7:16:48.5109220 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.5109220 AM	Length: 224, startime: 13602537, endtime: 13602537, seqnum: 0, connid: 0
7:16:48.5255318 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.5255318 AM	Length: 99, seqnum: 0, connid: 0
7:16:48.5263793 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.5263793 AM	Length: 11, startime: 13602539, endtime: 13602539, seqnum: 0, connid: 0
7:16:48.6064308 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6064308 AM	Length: 205, startime: 13602547, endtime: 13602547, seqnum: 0, connid: 0
7:16:48.6081812 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6081812 AM	Length: 128, seqnum: 0, connid: 0
7:16:48.6088565 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6088565 AM	Length: 224, startime: 13602547, endtime: 13602547, seqnum: 0, connid: 0
7:16:48.6092124 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6092124 AM	Length: 99, seqnum: 0, connid: 0
7:16:48.6095090 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6095090 AM	Length: 11, startime: 13602547, endtime: 13602547, seqnum: 0, connid: 0
7:16:48.6101142 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6101142 AM	Length: 205, startime: 13602547, endtime: 13602547, seqnum: 0, connid: 0
7:16:48.6136648 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6136648 AM	Length: 128, seqnum: 0, connid: 0
7:16:48.6142675 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6142675 AM	Length: 224, startime: 13602548, endtime: 13602548, seqnum: 0, connid: 0
7:16:48.6146254 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Receive	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6146254 AM	Length: 99, seqnum: 0, connid: 0
7:16:48.6148069 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Send	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6148069 AM	Length: 11, startime: 13602548, endtime: 13602548, seqnum: 0, connid: 0
7:16:48.6216828 AM	Duplicati.GUI.TrayIcon.exe	31960	0	TCP Disconnect	127.0.0.1:65353 -> 127.0.0.1:8200	SUCCESS	0.0000000	7:16:48.6216828 AM	Length: 0, seqnum: 0, connid: 0

Where else would I be seeing that? I posted the entire terminal output (after one redaction).

Duplicati.WindowsService.exe install works fine, and is how I forced the password in.
Rather absurd, but still less absurd than having to get a SYSTEM cmd.exe in order to do that.
Other very awkward solutions also exist, but I’m hoping for some non-painful way to do setup.

Tray icon can do the “First run setup” dialog, but I’m not sure the other configs can manage it.
As mentioned elsewhere, an Initial signin url doesn’t help get access if it can’t be seen.

Yes, as a service. By doing further testing, i was able to access the GUI.

In "C:\Program Files\Duplicati 2" run Duplicati.WindowsService.exe uninstall
In Task Manager kill "Duplicati.Server.exe" and "Duplicati.WindowsService.exe"

In "C:\Program Files\Duplicati 2" run Duplicati.WindowsService.exe install --webservice-interface=any --webservice-port=8200 --webservice-password=yourpassword

Run Duplicati icon for run Duplicati.GUI.TrayIcon.exe (without any parameters)

Oper your browser and go tu http://localhost:8200/, enter your password and it should work.

Start the service “Duplicati”

It doesn’t work, the TrayIcon doesn’t appear.

Even on prior releases, a running Server and TrayIcon have sometimes both gotten port 8200.

How do SO_REUSEADDR and SO_REUSEPORT differ? is an old post but gives the concept.

In 2.0.9.103 at least, I can crash TrayIcon if Server (in my case the service) is already running.

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Duplicati.GUI.TrayIcon.exe

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Unexpected error: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 401 (Unauthorized).)
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable`1 timeout)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternal[T](String method, String endpoint, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

TCP on port 8200 looks like this:

POST /api/v1/auth/signin HTTP/1.1
Host: localhost:8200
User-Agent: Duplicati-TrayIcon-Monitor/2.0.9.103
Content-Type: application/json; charset=utf-8
Content-Length: 265

{"SigninToken":"REDACTED"}

HTTP/1.1 401 Unauthorized
Content-Type: application/json
Date: Sun, 18 Aug 2024 11:44:26 GMT
Server: Kestrel
Cache-Control: no-cache,no-store
Expires: -1
Pragma: no-cache
Transfer-Encoding: chunked

28
{"Error":"Failed to sign in","Code":401}
0

The answer to which program gave the rejection is seen by watching both in Process Monitor.

Using duplicati in Docker, I now managed to launch it, using:

    environment:
      - DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES=*

However, when I limit the allowed hostnames, it no longer works correctly:

    environment:
      - DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES=duplicati.mydomain.duckdns.org

Duplicati starts, I get a login screen, but getting "The connection to the server is lost, attempting again in 0:11 …"-errors.

This seems to go a bit further:

    environment:
      - DUPLICATI__WEBSERVICE_ALLOWED_HOSTNAMES=duplicati.mydomain.duckdns.org;192.168.0.xxx

However, firefox gives the following error:

Firefox can’t establish a connection to the server at wss://duplicati.mydomain.duckdns.org/notifications?token=eyJhb_redacted. Error 403.

Note: duplicati runs behind a nginx server. Accessing via direct ip-access seems to work correctly. Nginx talks to duplicati via ip-address:

        set $upstream_app 192.168.0.xxx;
        set $upstream_port 8200;
        set $upstream_proto http;
        proxy_pass $upstream_proto://$upstream_app:$upstream_port;

I found the problem. The check for a hosted server was wrong, so it would crash while trying to get a token from the hosted server, regardless of (almost) all other settings.

It should be sufficient to pass the arguments to the service installer (or edit the arguments on the service afterwards):

Duplicati.WindowsService.exe INSTALL --webservice-password=<password>

If you do not specify a password, a random one will be generated, and you need to look into the logs to get the initial signin url, which will allow you to change the password.

That could also be the case, but in 2.0.9.103 (and .102) it will crash even with valid access to the database.

The “first run” dialog is part of the UI/Server, so it does not depend on TrayIcon.

That is a problem with service/daemon style setups: they have no interface, so you need to find the logs to get information.

Found the error. Fix is on the way :ambulance:

That should not be possible. Only one (operating system) process should be able to listen to a socket at any given time.

The SO_REUSEADR and SO_REUSEPORT is not for letting multiple processes use the same port, it is to handle a “dangling” socket that is not properly closed.

I have not been able to reproduce this. Is it caused by the two processes sharing the database, so they somehow override each other?

If you can see the request in the developer console that might help. In this case, I guess that it is the hostname check that fails? Based on the nginx config, I think the request is rewritten to use the IP and not the hostname, so the Duplicati server newer knows it had a hostname.

This sounds like an issue with the websockets. Either the problem is the hostname check or an nginx forwarding problem. I found this SO thread that explains how to enable websocket forwards for Nginx.

In case the problem is for the hostname check, the * should fix it. In the next canary, the hostname check automatically allows IP addresses as the hostname.

Thanks. It’s good to have the --no-hosted-server crash go. BTW the initial long list under

in the --webservice-interface=any runs were Duplicati self-crashes before I could try that.

More accurately, Duplicati is doing that, although I sort of see this as an explanation given the possibly unintentional (or does it help security?) loss of the “always allowed” in code and help.

Yes this is what two of us here have done. One drawback is that anyone can get the password.

I don’t think the Windows Services GUI allows this. The sc tool does, but quoting may be hard.

Which log? I don’t think a Windows service logs stdout, but this might work with Linux systemd.

Headless Linux servers may be even more of a challenge, as URL expects browsing localhost.

It was less of a problem before the new password requirement. Is that actually helping security?

Another hoop to jump through, and I’m not sure it’s even possible to satisfy. See remarks above.

Here’s what happens if TrayIcon is started, then Windows service is started. Note both got 8200:

C:\>netstat -ano | findstr 8200
  TCP    0.0.0.0:8200           0.0.0.0:0              LISTENING       35504
  TCP    127.0.0.1:8200         127.0.0.1:51990        ESTABLISHED     35504
  TCP    127.0.0.1:8200         127.0.0.1:51993        TIME_WAIT       0
  TCP    127.0.0.1:51982        127.0.0.1:8200         TIME_WAIT       0
  TCP    127.0.0.1:51990        127.0.0.1:8200         ESTABLISHED     35504
  TCP    192.168.56.1:51984     192.168.56.1:8200      TIME_WAIT       0

C:\>sc start duplicati

SERVICE_NAME: duplicati
        TYPE               : 10  WIN32_OWN_PROCESS
        STATE              : 2  START_PENDING
                                (NOT_STOPPABLE, NOT_PAUSABLE, IGNORES_SHUTDOWN)
        WIN32_EXIT_CODE    : 0  (0x0)
        SERVICE_EXIT_CODE  : 0  (0x0)
        CHECKPOINT         : 0x0
        WAIT_HINT          : 0x7d0
        PID                : 7232
        FLAGS              :

C:\>netstat -ano | findstr 8200
  TCP    0.0.0.0:8200           0.0.0.0:0              LISTENING       35504
  TCP    127.0.0.1:8200         0.0.0.0:0              LISTENING       26384
  TCP    127.0.0.1:8200         127.0.0.1:51993        TIME_WAIT       0
  TCP    127.0.0.1:8200         127.0.0.1:52002        ESTABLISHED     35504
  TCP    127.0.0.1:51982        127.0.0.1:8200         TIME_WAIT       0
  TCP    127.0.0.1:51990        127.0.0.1:8200         TIME_WAIT       0
  TCP    127.0.0.1:52002        127.0.0.1:8200         ESTABLISHED     35504
  TCP    192.168.56.1:52003     192.168.56.1:8200      TIME_WAIT       0

C:\>

image

Above is the opposite start order of the crash case, which apparently is hard to reproduce, but

Unlikely. Only the Windows service is running as SYSTEM. I suppose I can look for other clues.

EDIT 1:

Crash is reproducible here. I don’t know if --webservice-interface affects it, but try user first.

As me, a limited user:

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Unexpected error: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 401 (Unauthorized).)
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable`1 timeout)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternal[T](String method, String endpoint, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

As an elevated admin:

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Unexpected error: System.AggregateException: One or more errors occurred. (Response status code does not indicate success: 401 (Unauthorized).)
 ---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable`1 timeout)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternal[T](String method, String endpoint, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.ObtainAccessToken()
   at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.UpdateStatus(Boolean longpoll)
   at Duplicati.GUI.TrayIcon.HttpServerConnection..ctor(Uri server, String password, PasswordSource passwordSource, Boolean disableTrayIconLogin, Dictionary`2 options)
   at Duplicati.GUI.TrayIcon.Program.StartTray(String[] _args, Dictionary`2 options, HostedInstanceKeeper hosted, String password)

As SYSTEM (where a database clash with a SYSTEM Windows service seems probable):

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Unhandled exception. Duplicati.Library.Interface.UserInformationException: Server crashed on startup
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at System.ThrowHelper.Throw(String paramName)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.Server.Program.get_StatusEventNotifyer()
   at Duplicati.Server.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper.<>c__DisplayClass5_0.<.ctor>b__0(Object dummy_arg)
   --- End of inner exception stack trace ---
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper..ctor(String[] args)
   at Duplicati.GUI.TrayIcon.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.Net8.Program.Main(String[] args)

EDIT 2:

On the thinking that maybe a wildcard (0.0.0.0) and a specific (127.0.0.1) interface matters forcing TrayIcon to 127.0.0.1 (I can’t reconfigure in GUI because it won’t start) rejects with:

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>Duplicati.GUI.TrayIcon.exe --webservice-interface=loopback

C:\Duplicati\duplicati-2.0.9.103_canary_2024-08-15-win-x64-gui>fail: Microsoft.Extensions.Hosting.Internal.Host[11]
      Hosting failed to start
      System.IO.IOException: Failed to bind to address http://127.0.0.1:8200: address already in use.
       ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted.
       ---> System.Net.Sockets.SocketException (10048): Only one usage of each socket address (protocol/network address/port) is normally permitted.
         at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, Boolean disconnectOnFailure, String callerName)
         at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
         at System.Net.Sockets.Socket.Bind(EndPoint localEP)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportOptions.CreateDefaultBoundListenSocket(EndPoint endpoint)
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
         at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Infrastructure.TransportManager.BindAsync(EndPoint endPoint, ConnectionDelegate connectionDelegate, EndpointConfig endpointConfig, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.<>c__DisplayClass28_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         --- End of inner exception stack trace ---
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.EndpointsStrategy.BindAsync(AddressBindContext context, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(ListenOptions[] listenOptions, AddressBindContext context, Func`2 useHttps, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
         at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)
Unhandled exception. Duplicati.Library.Interface.UserInformationException: Server crashed on startup
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'provider')
   at System.ThrowHelper.Throw(String paramName)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
   at Duplicati.Server.Program.get_StatusEventNotifyer()
   at Duplicati.Server.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper.<>c__DisplayClass5_0.<.ctor>b__0(Object dummy_arg)
   --- End of inner exception stack trace ---
   at Duplicati.GUI.TrayIcon.HostedInstanceKeeper..ctor(String[] args)
   at Duplicati.GUI.TrayIcon.Program.Main(String[] _args)
   at Duplicati.GUI.TrayIcon.Net8.Program.Main(String[] args)

Using SO_REUSEADDR and SO_EXCLUSIVEADDRUSE (Microsoft) has quite a bit of advice, however I’m not familiar with Duplicati use, so will probably stop pursuit without more guidance.

From a Duplicati point of view, not being able to get 8200 should make it get 8300, shouldn’t it?

I think this is a quirk in how the sockets are assigned. For a machine with a single IP, you could probably listen to the same port on 0.0.0.0, 127.0.0.1 and the regular IP all at the same time.
I have not tested it, but I would assume the more specific IP will be routed before the fallback IP.

If you use the --webservice-interface= option, then this could explain how multiple hosted servers could co-exist on the same port. The hosted server connection that the TrayIcon makes does not take such a setup into account, so I would recommend using a different port for each.

Sorry, I did not intend to imply that it was you personally that was crafting the request.

If you need the TrayIcon to use a different hostname, you need to start it with --hosturl.
Something like:

> Duplicati.Server.exe --webservice-allowed-hostnames=hp4
> Duplicati.GUI.TrayIcon.exe --no-hosted-server --hosturl=http://hp4:8200

Without the --hosturl argument, the TrayIcon will always connect to http://localhost:8200 (with some minor adjustments for SSL + Port)

Yes, that is why the logic is to persist the password in the database. If you uninstall the service and reinstall without the argument, it will retain the password (and other arguments).

One way to persist the settings would be to run Duplicati.Server.exe to set the password, something like:

> Duplicati.Server.exe --webservice-password=<password> --webservice-allowed-hostnames=hp4

Wait for it to start up and report ready, then press CTRL+C to stop it.

If you have already installed the service, just make sure you stop it before doing the above, and start/install it afterwards.

You can use the argument --server-datafolder= to make the Duplicati.Server.exe point to the same folder as you are using for the service.

I am fairly certain that this is logged to Windows “Application Log”, but if not, it should be.

The daemon/service problem is not new, but before there was no option to use the log information to do an initial login, so you could not look for it.

I would argue that it increases the security quite a lot. Without a password, any local application (or website that can call localhost) could potentially set up a new backup to a destination of their choosing, or “restore” malicious content. This is even more of an issue if you are running as a service with admin privileges, as the call can be done by unprivileged users, granting privilege escalation without a password.

The downside you experience is only experienced for uses that want to run the server/service as stand-alone, not for users that just use the TrayIcon.

The latest canary seems to have fixed it. I still need to verify further, but initial tests are positive. No changes on nginx config needed apparently.

I tested it inadvertently by starting 2.0.7.1 to look at an issue previously only reported in 2.0.7.1.

C:\>netstat -ano | findstr 8200
  TCP    0.0.0.0:8200           0.0.0.0:0              LISTENING       36644
  TCP    127.0.0.1:8200         0.0.0.0:0              LISTENING       20484

PID 36644 is 2.0.9.104
PID 20484 is 2.0.7.1

Browsing to localhost:8200 gets 2.0.7.1.
Browsing to external interface 8200 by name or IP gets 2.0.9.104.
I’m not sure if this is 100% reliable (and haven’t read all the docs)

Super investigation! That means that the --server-listen-interface parameter is not used somehow. The correct version would be to default listen only on 127.0.0.1.

I don’t follow. I’m talking about (and tentatively supporting) your theory of connection routing.

Don’t follow this either. Assuming this means --webservice-interface, it sounds like it’s on

C:\>netstat -ano | findstr 8200
  TCP    0.0.0.0:8200           0.0.0.0:0              LISTENING       36644

which is 2.0.9.104 and which did not have that option, but does it have a specific default or use:

image

I haven’t tested old versions, but help text in general doesn’t talk hard-default vs. remembered, possibly from multiple sources, such as Settings or --webservice-interface on a prior start.

Here’s Process Explorer view showing the options, although it’s also on my terminal backscroll.

..\Duplicati.GUI.TrayIcon.exe --server-datafolder="C:\Duplicati\duplicati-2.0.9.104_canary_2024-08-21-win-x64-gui\RUN" --unencrypted-database --log-file="server.2.0.9.104_canary_2024-08-21-win-x64-gui.log" --log-level=profiling

Yes, my fault. The setting --webservice-interface is the one controlling what the server is listening on. Confusingly, the setting in the database is then called server-listen-interface.

Generally there are “three” values:

  • any: binds to 0.0.0.0
  • loopback: binds to 127.0.0.1
  • <ip>: binds to the specific IP

What you report is that 2.0.9.104 is bound to 0.0.0.0.
I was assuming this was without manually setting anything, but if you have any in the database, it suggests that you have at some point set it up for that.

If that is the case I think it is working “as designed”.

Yes, I think this area is a bit confusing, but generally:

  • The stored settings are the ones that have effect.
  • You can change stored settings via the UI, and they should take effect immediately, except for sever-listen-interface.
  • When you start Duplicati.Server.exe with arguments, they are stored before starting.
  • If no arguments are supplied to Duplicati.Server.exe you get the stored settings.

One thing that makes it confusing is that one could reasonably expect that removing the argument from the commandline would apply the default value for the argument, as that is how most other things work. However, using that logic makes it hard/impossible to allow changing settings from within the UI, as it cannot change the process that invokes Duplicati.Server.exe.