Release: 2.1.0.101 (Canary) 2024-12-06

kenkendk · December 6, 2024, 3:08pm

2.1.0.101_canary_2024-12-06

This release is a canary release intended to be used for testing.

Note that this build is using .NET8, unlike the 2.0.8.1 build

Breaking change from previous canary

The pCloud backend is updated to support subfolders.
Paths with / will now be treated as a folder structure on pCloud.
Previous version would treat folder/subfolder as a single foldername,
this version treats it as folder / subfolder.

Detailed list of changes:

Fixed a case where the TrayIcon would hang in detached mode
Improved OpenAPI description for filter groups
Fixed an issue with websocket requests through a proxy
Updated icons for Linux and dark-mode MacOS
Added logging of unobserved exceptions
Fixed a compatibility issue with FTP not correctly detecting explicit TLS
Fixed an issue where the quota was reported incorrectly
Removed the unused --use-block-cache option
Agent now has a dedicated register command
Sizes can now be supplied as kikibytes, eg. 20kib
Fixed supporting subfolders in pCloud
Added option to send http requests with relaxed TLS security
Agent can now be asked to retain the previous UI password
Updated SSH.NET to v2024.2.0

kevinfilipe · December 6, 2024, 4:33pm

Upgraded from 2.1.0.2_beta_2024-11-29. Invalid certificate warning disappeared. Thanks for looking at this!

Taomyn · December 9, 2024, 7:09am

Updated Win2022/Win11/Debian/Fedora/RPi-OS, all without issue and seem to be working fine so far - the Linux installs still need to perform scheduled backups but Windows were all fine last night.

Taomyn · December 11, 2024, 9:06am

Something I have noticed with the tray icon in Windows, if the service is not running the icon does not even appear. I have the service set to “Automatic (Delayed)” as I see no reason for it to slow the server boot down, but that means I can be logged in before it even starts. Can’t the tray icon simply show a “disconnected/not-running” icon but still show up?

Taomyn · December 11, 2024, 12:37pm

Another thing I noticed. Would it be possible for a “pause until resumed” to keep between Duplicati starts? I’m performing maintenance on my main backup storage server and so I went onto all my machines running Duplicati and paused the jobs. While waiting on the maintenance work I needed to reboot the server - when it came back up the missed jobs started. Now for the second time I have to cancel the maintenance (8hrs wasted), revert anything it did and lastly apply the VM snapshot made useless because new data got backed up, Yes I could have blocked the shares to prevent any access, but I shouldn’t have needed to because I had told Duplicati not to resume the pause. Thanks.

ts678 · December 11, 2024, 1:47pm

Anything worse in newer release? It appears you said this in 2019:

Duplicati Tray Icon Silently Dies with --no-hosted-server arg #3137 (This has not been resolved)

Testing this release and 2.0.8.1 from Command Prompt, both fail, but new one gives a message, possibly because it doesn’t do the old parent/child thing where the first process is just a launcher.

Watching in Wireshark, both try to connect. When it fails, they do 3 retries with 2 second spacing.

Taomyn · December 11, 2024, 2:04pm

Yes that’s basically it, I’d completely forgotten about that reported issue, and with all the server work I’ve been doing the past week the problem became apparent again.

From what you’ve mentioned, it shouldn’t just simply stop, it should display with the icon indicating the service isn’t contactable. I think that already happens if the icon was already running and service then stops.

kenkendk · December 11, 2024, 2:46pm

That should be fairly simple to handle. The connection could also drop while being connected, and this should also show “disconnected” instead of terminating the app.

It has a bit of retry, but will terminate if it cannot connect to the server.

I have created an issue for keeping the server paused between restarts.

ts678 · December 12, 2024, 4:12pm

Windows TrayIcon was stuck in its green backup-running state after running a small backup.

I’m not certain how long it has been that way, however current Wireshark shows it doing this:

GET /api/v1/serverstate?longpoll=true&lastEventId=9&duration=5m HTTP/1.1
Host: localhost:8200
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6InRyYXlpY29uIiwiZmFtIjoidGVtcG9yYXJ5IiwibmJmIjoxNzM0MDE3MzEwLCJleHAiOjE3MzQwMTgyMTAsImlzcyI6Imh0dHBzOi8vZHVwbGljYXRpIiwiYXVkIjoiaHR0cHM6Ly9kdXBsaWNhdGkifQ.vQXTZvPlf3ltSO1SmGPt4NpP9O6dICrj-FVK-vKmoRw
User-Agent: Duplicati-TrayIcon-Monitor/2.1.0.101


HTTP/1.1 401 Unauthorized
Content-Length: 0
Date: Thu, 12 Dec 2024 15:57:16 GMT
Server: Kestrel
WWW-Authenticate: Bearer error="invalid_token", error_description="The token expired at '12/12/2024 15:43:30'"

This traffic is from a TrayIcon port as seen below in Process Explorer, and I closed other tabs.

This TrayIcon has not been up long. I started it for a short backup before a move to 2.1.0.102.

kenkendk · December 12, 2024, 9:29pm

That is “normal” in that the access token can expire, and the TrayIcon will just request a new token when this happens. If the trayicon fails to get a valid token, it should terminate after a few retries.

Is the UI showing that the backup is still running (stuck), or is it just the trayicon that hangs? Does the trayicon seem unresponsive?

ts678 · December 12, 2024, 10:10pm

It’s been failing this way for hours. P.S. meaning as said. I don’t have a tally on token get tries.

GUI is fine, looks idle, runs jobs. TrayIcon can bring up its menu. Unsurprisingly, a Pause can’t, however Open works (guessing it just goes to Windows without having to work with the server).

TrayIcon is trying and failing with the server every 10 seconds. Failure isn’t shown on terminal.

EDIT:

It doesn’t, unless it’s supposed to happen far later.

Note that there was the oddity where refresh wasn’t being answered, but maybe got sent wrong.
Could some unanswered refresh from early on stop new “request a new token” from happening?

kenkendk · December 13, 2024, 1:03pm

I have not heard that happen before. Is this standalone trayicon or with the hosted server?

It should not be hours at least.

I think you are on the right track: it is some unexpected state where it keeps retrying for some reason.

ts678 · December 13, 2024, 1:17pm

Standalone in my usual scripted launch configuration from a .zip install with more logging, etc.

I tried looking into the code last night, got lost, but have a debugger ready if you want to advise.

kenkendk · December 13, 2024, 2:15pm

The logic for keeping the connection is in HttpServerConnection.cs so if it somehow fails to keep a connection, this is where it should eventually fail.

It is also possible that the HttpServerConnection does indeed fail, but somehow does not manage to inform the TrayIcon about the exit. This event is in Duplicati.GUI.TrayIcon → Program.cs → StartTray() → Action shutdownEvent = ....

ts678 · December 15, 2024, 1:28am

I’m not sure how far I’ll get chasing those, but. Initial finding is that Debug output shows:

19:34:24.147 Exception thrown: 'System.InvalidOperationException' in Microsoft.IdentityModel.Protocols.dll
19:34:24.152 Additional information: IDX20803: Unable to obtain configuration from: 'https://duplicati/.well-known/openid-configuration'. Will retry at '12/15/2024 12:36:11 AM +00:00'. Exception: 'System.IO.IOException: IDX20804: Unable to retrieve document from: '[PII of type 'System.String' is hidden. For more details, see https://aka.ms/IdentityModel/PII.]'.
19:34:24.154  ---> System.Net.Http.HttpRequestException: No such host is known. (duplicati:443)
19:34:24.156  ---> System.Net.Sockets.SocketException (11001): No such host is known.
19:34:24.157    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
19:34:24.158    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
19:34:24.159    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
19:34:24.160    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.161    --- End of inner exception stack trace ---
19:34:24.162    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.164    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.165    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.166    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
19:34:24.167    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
19:34:24.168    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
19:34:24.169    at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.170    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.171    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
19:34:24.172    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.SendAsyncAndRetryOnNetworkError(HttpClient httpClient, Uri uri)
19:34:24.173    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.175    --- End of inner exception stack trace ---
19:34:24.176    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.176    at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)
19:34:24.177    at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)'.
19:34:24.179 Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.dll
19:34:24.180 Additional information: IDX20803: Unable to obtain configuration from: 'https://duplicati/.well-known/openid-configuration'. Will retry at '12/15/2024 12:36:11 AM +00:00'. Exception: 'System.IO.IOException: IDX20804: Unable to retrieve document from: '[PII of type 'System.String' is hidden. For more details, see https://aka.ms/IdentityModel/PII.]'.
19:34:24.181  ---> System.Net.Http.HttpRequestException: No such host is known. (duplicati:443)
19:34:24.182  ---> System.Net.Sockets.SocketException (11001): No such host is known.
19:34:24.183    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
19:34:24.184    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
19:34:24.185    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
19:34:24.186    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.187    --- End of inner exception stack trace ---
19:34:24.188    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.189    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.190    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.191    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
19:34:24.194    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
19:34:24.195    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
19:34:24.196    at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.197    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.198    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
19:34:24.199    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.SendAsyncAndRetryOnNetworkError(HttpClient httpClient, Uri uri)
19:34:24.199    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.200    --- End of inner exception stack trace ---
19:34:24.201    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.202    at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)
19:34:24.203    at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)'.
19:34:24.204 Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.dll
19:34:24.206 Additional information: IDX20803: Unable to obtain configuration from: 'https://duplicati/.well-known/openid-configuration'. Will retry at '12/15/2024 12:36:11 AM +00:00'. Exception: 'System.IO.IOException: IDX20804: Unable to retrieve document from: '[PII of type 'System.String' is hidden. For more details, see https://aka.ms/IdentityModel/PII.]'.
19:34:24.207  ---> System.Net.Http.HttpRequestException: No such host is known. (duplicati:443)
19:34:24.208  ---> System.Net.Sockets.SocketException (11001): No such host is known.
19:34:24.209    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
19:34:24.210    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
19:34:24.211    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
19:34:24.212    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.213    --- End of inner exception stack trace ---
19:34:24.213    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
19:34:24.214    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.215    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.219    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
19:34:24.219    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
19:34:24.220    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
19:34:24.221    at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.222    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
19:34:24.223    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
19:34:24.224    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.SendAsyncAndRetryOnNetworkError(HttpClient httpClient, Uri uri)
19:34:24.225    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.226    --- End of inner exception stack trace ---
19:34:24.227    at Microsoft.IdentityModel.Protocols.HttpDocumentRetriever.GetDocumentAsync(String address, CancellationToken cancel)
19:34:24.228    at Microsoft.IdentityModel.Protocols.OpenIdConnect.OpenIdConnectConfigurationRetriever.GetAsync(String address, IDocumentRetriever retriever, CancellationToken cancel)
19:34:24.229    at Microsoft.IdentityModel.Protocols.ConfigurationManager`1.GetConfigurationAsync(CancellationToken cancel)'.
19:34:24.230 Exception thrown: 'Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException' in Microsoft.IdentityModel.Tokens.dll
19:34:24.231 Additional information: IDX10223: Lifetime validation failed. The token is expired. ValidTo (UTC): '12/12/2024 3:43:30 PM', Current time (UTC): '12/15/2024 12:34:24 AM'.
19:34:24.233 Exception thrown: 'Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException' in System.Private.CoreLib.dll
19:34:24.234 Additional information: IDX10223: Lifetime validation failed. The token is expired. ValidTo (UTC): '12/12/2024 3:43:30 PM', Current time (UTC): '12/15/2024 12:34:24 AM'.
19:34:24.241 Exception thrown: 'System.Net.Http.HttpRequestException' in System.Net.Http.dll
19:34:24.245 Additional information: Response status code does not indicate success: 401 (Unauthorized).
19:34:24.258 Exception thrown: 'System.Net.Http.HttpRequestException' in System.Private.CoreLib.dll
19:34:24.260 Additional information: Response status code does not indicate success: 401 (Unauthorized).

That sort of output repeats every 10 seconds.

FWIW breakpoints in that area of code aren’t going off (if I set it up right).

I’m looking at LongPollRunner. It tries UpdateStatus, but gets exception:

{System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequestInternalAsync[T](String method, String endpoint, String body, Nullable1 timeout) at Duplicati.Library.Utility.Utility.Await[T](Task1 task)
at Duplicati.GUI.TrayIcon.HttpServerConnection.PerformRequest[T](String method, String urlfragment, String body, Nullable`1 timeout)
at Duplicati.GUI.TrayIcon.HttpServerConnection.LongPollRunner()}

EDIT 1:

Perhaps not surprisingly, 2.1.0.102 does this too:

GET /api/v1/serverstate?longpoll=true&lastEventId=29&duration=5m HTTP/1.1
Host: localhost:8400
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6InRyYXlpY29uIiwiZmFtIjoidGVtcG9yYXJ5IiwibmJmIjoxNzM0MTg3MDEyLCJleHAiOjE3MzQxODc5MTIsImlzcyI6Imh0dHBzOi8vZHVwbGljYXRpIiwiYXVkIjoiaHR0cHM6Ly9kdXBsaWNhdGkifQ.AnCKi6ixwikjwLQ_ElOXsA_8eKPU80vzBW9l3UNsLwA
User-Agent: Duplicati-TrayIcon-Monitor/2.1.0.102


HTTP/1.1 401 Unauthorized
Content-Length: 0
Date: Sun, 15 Dec 2024 12:59:15 GMT
Server: Kestrel
WWW-Authenticate: Bearer error="invalid_token", error_description="The token expired at '12/14/2024 14:51:52'"

Run circumstances are a bit better known. Process started at 9:36:35 AM 12/14/2024.
Computer slept overnight. Today I wanted to Quit, but wanted to ensure I got right one.
Put it in pause in Web UI, went to look for paused icon. None, so looked in Wireshark.
I’m not sure when it first broke. This Duplicati has no jobs and hasn’t been used much.

C:\>netstat -ano | findstr 8400
  TCP    0.0.0.0:8400           0.0.0.0:0              LISTENING       33900
  TCP    [::]:8400              [::]:0                 LISTENING       33900
  TCP    [::1]:8400             [::1]:61522            ESTABLISHED     33900
  TCP    [::1]:8400             [::1]:64519            ESTABLISHED     33900
  TCP    [::1]:61522            [::1]:8400             ESTABLISHED     33900
  TCP    [::1]:64519            [::1]:8400             ESTABLISHED     9568

C:\>

PID 9568 is Edge. I “think” this Duplicati has only had one tab so far, unlike some others.

kenkendk · December 16, 2024, 9:55pm

ts678:

19:34:24.147 Exception thrown: 'System.InvalidOperationException' in Microsoft.IdentityModel.Protocols.dll
19:34:24.152 Additional information: IDX20803: Unable to obtain configuration from: 'https://duplicati/.well-known/openid-configuration'.

That is unrelated but a configuration error in the JWT setup. I have a PR that removes this call.

That should bring down the TrayIcon. I have created an issue for tracking this.

ts678 · December 17, 2024, 3:56am

The issue talks about detached mode. If that means --no-hosted-server, mine aren’t.
Issue also doesn’t call for bringing down the TrayIcon, but for showing disconnect icon.
While there’s been talk of wanting that, e.g. if service start delays, that seems different.

I’m not sure either an exit or an icon is a good response to an unwanted 401 response.

EDIT:

It’s still doing that (after I close the two tabs to reduce the noise), and on same process.

kenkendk · December 18, 2024, 7:21am

Ok, I assumed it was detached mode, as most issues reported in this area are for such a setup.

True, but I guess that is two ways of handling the same: exit or show disconnected.