Release: 2.1.0.120 (Canary) 2025-06-24

[ts678]

@kenkendk

Where is the “later C# read” ? All code that reads an enum option from a string should use the helper method ParseEnum to make sure we always parse the same way.

Could it be implicit with an operator overload on == and !=? If not, lots are omitted.
GitHub search FullRemoteVerification then see if you like TestHandler.cs and LocalTestDatabase.cs. If you dislike those, then look at some of the similar enums.

@kenkendk

I understand the gripe, but I think the actual option name is for power users, and has little value for the regular person.

It has huge value if the regular person ever wants to consult with others on some option because otherwise all one can do is post help text or images around in many languages. Having to use the translation files in source (for text) or Google Lens (images) is terrible.

@kenkendk

I think the option to have the ... edit as text would solve it, by showing the actual option names.

Agreed, although it’s a bit out of the way, but hopefully such a need isn’t all that common. Option names are also useful as the initial picker mechanism on the unsorted options list.

Interestingly, I found that options already in use are removed from the picker, so one can type a known name and if it’s not there, then try to find it from its help in the options set…

It will be interesting to see what feedback comes from regular users on what’s nice or not. Problem is that it might be too late then, which is why I keep hoping a UX expert is in this.

[kenkendk]

@ts678

Could it be implicit with an operator overload on == and !=? If not, lots are omitted.
GitHub search FullRemoteVerification then see if you like TestHandler.cs and LocalTestDatabase.cs. If you dislike those, then look at some of the similar enums.

That happens after the parsing. The Options.cs class is constructed so it exposes the parsed values. (As a side note, it is parsing on each request, which is not efficient).

See:

Duplicati/Library/Main/Options.cs

This means that this line:

Duplicati/Library/Main/Operation/TestHandler.cs

Is doing a compare of the parsed enum, so there is no case-sensitivity issues (in fact it is an integer compare when compiled).

@ts678

It has huge value if the regular person ever wants to consult with others on some option because otherwise all one can do is post help text or images around in many languages. Having to use the translation files in source (for text) or Google Lens (images) is terrible.

I see. So you are saying that it is hard to find the option mentioned (especially if UI is localized).
I will take it up with the designer.

@ts678

which is why I keep hoping a UX expert is in this.

We do have an expert onboard for the UI/UX part

[Taomyn]

@kenkendk

@Taomyn

I had to stop a backup because it was trying to back up some unexpected files (caused by a mount), so after excluding them in the job and restarted, the next backup completed but had these warnings:

That sounds like a bug somewhere. The logic is that such failed uploads should be marked is “uploading” in the database, and since the backup did not shut down cleanly, the next run should remove these files as they are not fully uploaded. The error message you see indicates that they somehow got the state “uploaded” or “verified” in the database.

Could you make a bugreport database and PM it to me? Then I will try to figure out where the transition happened.

@Taomyn

What’s the best thing to do with this?

Since the files are damaged, you should delete them. They cannot really be read due to encryption. After removal, you need to run purge-broken-files.

I didn’t see this until after I upgraded to .122 just now, so I hope this hasn’t made things worse. I tried the create bug report and got this:

Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'cannot VACUUM from within a transaction'.
   at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
   at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
   at System.Data.Common.DbCommand.ExecuteNonQueryAsync(CancellationToken cancellationToken)
--- End of stack trace from previous location ---
   at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQueryAsync(SqliteCommand self, Boolean writeLog, String cmd, Dictionary`2 values, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQueryAsync(SqliteCommand self, String cmdtext, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Database.LocalBugReportDatabase.Fix(CancellationToken token)
   at Duplicati.Library.Main.Database.LocalBugReportDatabase.Fix(CancellationToken token)
   at Duplicati.Library.Main.Operation.CreateBugReportHandler.RunAsync()
   at Duplicati.Library.Main.Operation.CreateBugReportHandler.RunAsync()
   at Duplicati.Library.Utility.Utility.Await(Task task)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Func`3 method)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, Func`3 method)
   at Duplicati.Library.Main.Controller.CreateLogDatabase(String targetpath)
   at Duplicati.Server.Runner.RunInternal(Connection databaseConnection, EventPollNotify eventPollNotify, INotificationUpdateService notificationUpdateService, IProgressStateProviderService progressStateProviderService, IApplicationSettings applicationSettings, IRunnerData data, Boolean fromQueue)

Should I still delete the files?

[kenkendk]

@Taomyn

I tried the create bug report and got this:

Arh, another casualty from the SQLite upgrade. I have a fix ready.

@Taomyn

Should I still delete the files?

To progress, then yes. If you can wait a bit, I would still very much like to get the bugreport before the files are deleted. Can you use 2.1.0.120 to create the bugreport if you cannot wait?

[Taomyn]

I can wait, I presume the fix will be in .123?

[ts678]

@Taomyn

got another error that mentioned disk space, but that is fine

What partition is fine? A bug report uses tempdir space (maybe three times the DB size).

[Taomyn]

This is one bit I hate with Linux, Fedora + LVM especially. You’re probably right as I think the /tmp is probably not large enough:

Filesystem                                        Size  Used Avail Use% Mounted on
/dev/mapper/fedora_fedora-root                     99G   37G   63G  37% /
devtmpfs                                          412M     0  412M   0% /dev
tmpfs                                             5.0G   84K  5.0G   1% /dev/shm
efivarfs                                          256K   83K  168K  34% /sys/firmware/efi/efivars
tmpfs                                             2.0G  2.4M  2.0G   1% /run
tmpfs                                             1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
tmpfs                                             5.0G  890M  4.1G  18% /tmp
/dev/sda2                                        1014M  275M  740M  28% /boot
/dev/sdb                                          200G  152G   49G  76% /srv/nextcloud
/dev/sda1                                         599M  7.5M  592M   2% /boot/efi
tmpfs                                             1.0M     0  1.0M   0% /run/credentials/systemd-resolved.service
tmpfs                                             1.0M     0  1.0M   0% /run/credentials/getty@tty1.service
tmpfs                                            1008M  8.0K 1008M   1% /run/user/0

Just no clue about extending it if that’s what I must do - my guess, I need to extend the partition for fedora_fedora-root then extend one of the tmpfs

[ts678]

@Taomyn

Just no clue about extending it if that’s what I must do

It’s easier to tell Duplicati to use a different tempdir. There’s an option by that name in Advanced options, but I’m not certain it works exactly as it says, especially with Linux.

@obones

The --tempdir option worked fine

gives me hope, and you can read the rest of the discussion there on difficulties with this. There was at least one case I found where it seemed like SQLite had set itself up before request for change was processed. Maybe prior action already had gotten it initialized…

[Taomyn]

Thanks, I was able to do with the --tempdir parameter and got the file.

FYI I could have used this to temporarily change the size of /tmp

mount -o remount,size=10G /tmp

Which on reboot would fall back to 5G

But I’m wondering if perhaps having just 5G could have caused some issues - this is the biggest of my Linux servers, and on all my Windows servers I used the same parameter to point Duplicati to a separate drive mostly for performance. For now, I have left it pointing at a separate directory on another disk but I might do the same as Windows and create a separate disk for temporary storage like this.

[ts678]

@kenkendk

There is also a more agressive command (named abort internally) that will instead attempt to stop any processing immediately, potentially leaving partial files on the storage.

@kenkendk

The “Stop now” is the same as abort. It has a skull icon in ngclient, and you need to click “stop” before you can see that button.

Is the ngax “Stop now” supposed to error

instead of the ngax “Stop after the current file”

Often users think “A task was canceled” means they did something. It’s true here though.
I’m actually testing on 2.1.0.124, but the stop issue first began discussion roughly here…

@kenkendk

For the “soft stop” a new filelist will be uploaded, if any new data was encountered, as the backend will otherwise store “dangling” data until the next backup can upload a (synthetic) filelist.

If ngclient stop button is supposed to be like ngax “Stop now”, it seems to do a lot more, such as uploading a partial dlist file, and it also doesn’t need any changes to cause that.

2.1.0.122 Canary ngclient stop uploads a short dlist file

EDIT 1:

As evidence of no changes, I don’t have upload-unchanged-backups set, so no backup version and no dlist file is created on a normal repeat backup without any stops before it.

[kenkendk]

@ts678

Is the ngax “Stop now” supposed to error

That is up for discussion. It shows the error because internally an error is thrown that activates the shutdown. Since the user is responsible for the error being thrown in this case, it is arguably “as intended” an could be shown differently.

@ts678

Often users think “A task was canceled” means they did something. It’s true here though.

This is just the .Net message for cancelling a task. It does not have any idea what cancelled it. The task cancellation trigger could be a timer, a side-effect from something else stopping, the user, etc.

We can replace the message with another sentence, but I think the underlying issue is that we are lacking information about the cause of the cancellation. We could (do?) keep track of the user issued cancellation and show something different here.

@ts678

If ngclient stop button is supposed to be like ngax “Stop now”, it seems to do a lot more,

The text “Stop now” text seems to be on the commandline UI? My description until now assumed this was considering the status bar “Stop” button(s).

For some reason there is no soft-stop for the commandline in neither ngax nor ngclient. Both frontends implement the stop here as abort, so they should do the same at least.

[ts678]

@kenkendk

My description until now assumed this was considering the status bar “Stop” button(s).

Testing .120 again, it looks like I meant “Terminate”, basically the new faster stop button option that isn’t “Stop after the current file” and that has the (new, I think) “A task was cancelled” user message.

EDIT 1:

I had done an early quick test of .125, not seen the problem, so tested .120 (which we’re in now).

Going back to .125 for a test with a longer backup, it’s kind of hung for over 25 minutes now, with

This was actually Terminate, but status message has been wrong even when it was “Stop now”:

Choose ‘Stop now’ but status then shows ‘Stopping after current file’

Profiling log ends with:

2025-08-05 07:25:26 -04 - [Retry-Duplicati.Library.Main.Backend.Handler-RetryPut]: Operation Put with file duplicati-b71e54ef3383042c3975f1912f1747b15.dblock.zip attempt 1 of 5 failed with message: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Duplicati.Library.Main.Operation.Common.TaskControl.TransferRendevouz()
   at Duplicati.Library.Utility.Utility.Await[T](Task`1 task)
   at Duplicati.Library.Main.Backend.BackendManager.ProgressHandler.HandleProgress(Int64 pg, String filename)
   at Duplicati.Library.Utility.ProgressReportingStream.ReadImplAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Duplicati.Library.Utility.Utility.CopyStreamAsync(Stream source, Stream target, Boolean tryRewindSource, CancellationToken cancelToken, Byte[] buf)
   at Duplicati.Library.Backend.File.PutAsync(String targetFilename, Stream sourceStream, CancellationToken cancelToken)
   at Duplicati.Library.Main.Backend.BackendManager.PutOperation.PerformUpload(IBackend backend, String hash, Int64 size, CancellationToken cancelToken)
   at Duplicati.Library.Main.Backend.BackendManager.PutOperation.ExecuteAsync(IBackend backend, CancellationToken cancelToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.Execute[TResult](PendingOperation`1 op, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.Execute(PendingOperationBase op, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.ExecuteWithRetry(PendingOperationBase op, CancellationToken cancellationToken)
```

EDIT 2:

If I duplicate the Edge tab to see if I can get a different look, I can get the stuck status bar to say:

EDIT 3:

In developer tools, this is a “Terminate” followed by a “Stop after the current file”. Still stuck here.

Profiling log did get a new line, probably from the ‘stop’ based on time, but without further activity.

2025-08-05 08:04:18 -04 - [Verbose-Duplicati.Library.Main.Controller-CancellationRequested]: Cancellation Requested

[kenkendk]

@ts678

Going back to .125 for a test with a longer backup, it’s kind of hung for over 25 minutes now, with

Is this the UI that is hanging? Or is Duplicati actually hanging for that long?

[ts678]

@kenkendk

Is this the UI that is hanging? Or is Duplicati actually hanging for that long?

It’s closer to Duplicati hanging, but it’s only the job. The UI seems responsive.

Release: 2.1.0.125 (Canary) 2025-07-15 has followup, including a .124 repro.

Fine with me if you want to keep talk here, since exact break point isn’t known.

[ts678]

@ts678

2025-06-26 13:39:03 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The CancellationTokenSource has been disposed.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The CancellationTokenSource has been disposed.)
 ---> System.ObjectDisposedException: The CancellationTokenSource has been disposed.
   at Duplicati.Server.Runner.<>c__DisplayClass16_2.<<RunInternal>b__3>d.MoveNext()
   --- End of inner exception stack trace ---

has now also been reported by another user in similar (maybe not identical) form as

@jberry02

Jul 18, 2025 3:30 AM: A Task’s exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The CancellationTokenSource has been disposed.)

This morning my nearly-three-week-uptime 2.1.1.0 had a flurry of these as final words, although the circumstances of its death are mysterious. No crashlog. No events found.

2025-08-07 05:01:45 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 05:47:06 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 06:00:06 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 07:06:18 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 07:20:03 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
2025-08-07 07:35:11 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
2025-08-07 07:58:42 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.

I wasn’t actually up at 5, but system tends to wake itself at 3 for Windows Update tasks.
7:20 is the scheduled backup run. The run-script-after run was about 7:25:11 to 7:39:55.

2025-08-07 03:00:51 -04 - [Verbose-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketConnectionTimeout]: WebSocket connection timed out after 60 seconds.
2025-08-07 05:01:45 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 05:47:06 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 06:00:06 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 07:06:18 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 07:20:03 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:20:04 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 07:20:11 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started
2025-08-07 07:20:11 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Controller-RunBackup]: Starting - Running Backup
...
2025-08-07 07:25:07 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Controller-RunBackup]: Running Backup took 0:00:04:56.003
2025-08-07 07:25:07 -04 - [Verbose-Duplicati.Library.SQLiteHelper.SQLiteLoader-CustomSQLiteOption]: Setting custom SQLite option 'cache_size=-333824'.
2025-08-07 07:25:07 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteReader]: Starting - ExecuteReader: SELECT "ID", "Timestamp" FROM "Operation" ORDER BY "Timestamp" DESC LIMIT 1
2025-08-07 07:25:07 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteReader]: ExecuteReader: SELECT "ID", "Timestamp" FROM "Operation" ORDER BY "Timestamp" DESC LIMIT 1 took 0:00:00:00.161
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:25:08 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.WriteAsync(T value, ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-08-07 07:25:10 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: Starting - ExecuteNonQuery: PRAGMA optimize
2025-08-07 07:25:10 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-08-07 07:35:11 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---
2025-08-07 07:39:56 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Backup has completed
2025-08-07 07:58:42 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---

Job database time is 7:25:42 and server database time is 8:13:03. I’m not sure when process actually went down. I noticed it later when I went to use it and it wasn’t there.

I wouldn’t have intentionally Quit it, as this was my uptime leader. Lots of threads and handles accumulated – and I don’t know if that could be a factor in it getting in trouble.

[kenkendk]

@ts678

This morning my nearly-three-week-uptime 2.1.1.0 had a flurry of these as final words

The UnobservedTaskException is a bit of a pain, because they happen when some detached code has an error, so the context is not really known.

From the ones there, I do not see any dangerous ones. Attempting to access a disposed object is usually done during some cleanup, and the retired channel is also expected when shutting down some code.

I don’t think these are the cause of the crash, but they are most likely lingering and then being cleaned up due to another crash that brings the server down.

@ts678

2025-08-07 05:01:45 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.)
 ---> System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'Microsoft.Win32.SafeHandles.SafeWaitHandle'.
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation)
   at System.Threading.WaitHandle.ObtainSafeWaitHandles(ReadOnlySpan`1 waitHandles, Span`1 safeWaitHandles, Span`1 unsafeWaitHandles)
   at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)
   at Duplicati.GUI.TrayIcon.HttpServerConnection.<>c__DisplayClass47_0`1.<PerformRequestInternalAsync>b__0()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
   --- End of inner exception stack trace ---

This one points to the TrayIcon’s connection to the server. The wait handle is disposed when the server is closing, so this indicates that the server was stopping (for some reason) and the TrayIcon does not handle this gracefully. The code in question was updated for 2.1.1.100, so at least that error message should now be gone, but it does not explain why the server stopped.