Release: 2.1.1.103 (Canary) 2025-09-25

kenkendk · September 25, 2025, 3:23pm

2.1.1.103_canary_2025-09-25

This release is a canary release intended to be used for testing.

Changes in this versions

This version is a bugfix release for some issues discovered with the previous canary build.

Database update to v9 (also in previous version)

This version updates the server settings database to v9 to add a new column to the Backup table.
The database tool can downgrade without data loss (assuming that external configs were not loaded).

Detailed list of changes:

Fixed issue with invariant formatting causing crashes during backup
Added Administrator permissions to data folders on Windows
Fixed an issue with the “Test connection” button after passwords are masked

ngclient changes:

Updated localizations
Updated API calls to support masked passwords for test connection

jberry02 · September 25, 2025, 4:21pm

Installed on Windows 10. Service startup cannot find:

C:\Program Files\Duplicati 2\Duplicati.Windows.Service.exe

The path exists, but there are only 3 files in the directory. Currently searching to see if the file exists somewhere else…..

And never mind - accidentally downloaded and installed the x84 instead of the x64 kit.

ts678 · September 25, 2025, 4:36pm

This fixed my new 2.1.1.102 problems with backup, recreate, etc. Thanks for fast fixing.

jberry02 · September 25, 2025, 4:57pm

1 Windows 10 desktop, 2 Fedora 42 laptops upgraded. All backups triggered manually, all ran without any apparent issues.

Thanks for the quick bugfix!

f1assistance · September 25, 2025, 6:24pm

I’m unable to select files to restore in ver. 102 and 103. I had to ‘Use old UI’ in both which allowed select files? I didn’t try to restore files in ver. 101…

ts678 · September 25, 2025, 8:24pm

What happens step by step? Ultimately a select is a left click, and file name goes bold.
Working for me in this release, testing Windows 10 backup in Edge or Firefox browser.

f1assistance · September 25, 2025, 11:54pm

There are no files shown to cllck, nothing, but they’re shown with the old UI…

ts678 · September 26, 2025, 12:13am

Nothing where the file tree should be? Did version picker offers versions?

If you open browser developer tools (often on F12 key), I think the versions are from the list-filesets, and the list folder show the folders and the contents of first folder.

You can check to see if requests seem to be made and answered, while waiting for devs.

If this is a “Direct restore” or anything other than a plain old Restore from job, please say.

Taomyn · September 26, 2025, 7:07am

Fixed my issue on Windows, thanks but now on Debian machine I can’t run a repair.

 The operation Repair has failed => You have attempted to change a passphrase to an existing backup, which is not supported. Please configure a new clean backup if you want to change the passphrase. 
 
 
 ErrorID: PassphraseChangeNotSupported 
 You have attempted to change a passphrase to an existing backup, which is not supported. Please configure a new clean backup if you want to change the passphrase. 
 Return code: 100

The password got redacted, so I tried entering it but it said the same. I switched to the old GUI and it worked fine.

BTW, any chance that Repair could be a standard command now without needing to go through the command-line menu? I can’t be alone in needing to do this almost daily as my installations gradually fixes past backup data?

kenkendk · September 26, 2025, 8:18am

That sounds like something we need to fix ASAP.

Can you share:

What operating system?
What browser brand and version?
Do you see any errors with the Developer tools enabled (F12 on Linux/Windows, CMD+Alt+I on MacO).
Anything special with the paths?

It is that already. Pick the “Database” menu from the overview, then there should be a “Repair” button:

That is the new masking that needs to be dealt with for the commandline page. I will fix it.

kenkendk · September 27, 2025, 9:00am

2 posts were split to a new topic: Missing files on remote

ts678 · September 26, 2025, 10:53pm

Compared with 2.1.1.102, ngax “Local folder or drive” now has “Server and port”?

It also provides no way to browse folders. Here’s 2.1.1.102 in “Manually type path”:

kenkendk · September 27, 2025, 8:59am

Ouch. A small cleanup wiped all the editors, so no matter what destination you choose, it uses a generic view.

I have a fix for it ready.

ts678 · October 2, 2025, 5:54pm

Compared to 2.1.2.0_beta_2025-08-20, job logs tend to go missing instead of giving error, which of course makes it harder for users to debug, especially since GUI goes to a job log instead of server log which usually has the error. On occasion, there’s nothing there either.

I had some networking trouble this morning, and a whole lot of B2 failures made no job log.

My note is about destination-side errors such as timeout, down server, bad password, etc. Tested a variety of types, but not systematically. I can’t speak for treatment of other errors.

kenkendk · October 3, 2025, 11:59am

Is this a new thing with 2.1.1.100+ ?
If so, it might be related to the database update, where it rolls back a transaction and loosing the logs in the process.

ts678 · October 3, 2025, 12:19pm

Not sure when it first began. First test:

Test 2.1.0.125_canary_2025-07-15:
Disconnect Wi-Fi and try B2 backup Verify files.

Red error popup:
No such host is known. (api.backblazeb2.com:443)

No job log.

About --> Show log --> Stored has error (twice):
Oct 3, 2025 8:09 AM: Error in worker
Oct 3, 2025 8:09 AM: Failed while executing Verify "HP4 clone 12 Backblaze B2" (id: 1)

2.1.1.103 just did an unwanted log loss on what is not a destination accessibility error.

Found 1 remote files that are not recorded in local storage. This can be caused by having two backups sharing a destination folder which is not supported. It can also be caused by restoring an old database. If you are certain that only one backup uses the folder and you have the most updated version of the database, you can use repair to delete the unknown files.

and this made no job log, but did make the double server log entry, somewhat uselessly, because it doesn’t name that file. I’m digging in profiling log, but what users keep those?

EDIT 1:

2.1.0.120_canary_2025-06-24 makes a log, shown specially in ngax, less so in ngclient:

ts678 · October 4, 2025, 1:12am

“Found 1 remote files that are not recorded in local storage” looks like a dblock delete from compact that failed, but got Deleted State anyway. DeleteGraceTimer held for 2 hours with

2025-10-02 08:42:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes (9.92 MiB)
2025-10-02 08:42:27 -04 - [Retry-Duplicati.Library.Main.Backend.Handler-RetryDelete]: Operation Delete with file duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes attempt 1 of 0 failed with message: An error occurred while sending the request.
2025-10-02 08:42:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Failed: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes (9.92 MiB)
2025-10-02 08:43:05 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
2025-10-02 08:45:42 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
2025-10-02 08:52:43 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
2025-10-03 07:20:09 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
2025-10-03 07:23:08 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
C:     2025-10-03 07:20:09 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes,
2025-10-03 07:24:52 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-ExtraUnknownFile]: Extra unknown file: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes
C:     2025-10-03 07:24:52 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-ExtraUnknownFile]: Extra unknown file: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes,
C: 2025-10-03 07:24:52 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-ExtraUnknownFile]: Extra unknown file: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes

Profiling log view of failure-treated-as-success (?) is like below. I option to allow 0 retries.

2025-10-02 08:42:27 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Backend.Handler-RemoteOperationDelete]: Starting - RemoteOperationDelete
2025-10-02 08:42:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes (9.92 MiB)
2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestStart]: [System.Net.Http] Event: RequestStart
  scheme              : https
  host                : api001.backblazeb2.com
  port                : 443
  pathAndQuery        : /b2api/v1/b2_delete_file_version
  versionMajor        : 1
  versionMinor        : 1
  versionPolicy       : 0

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestHeadersStart]: [System.Net.Http] Event: RequestHeadersStart
  connectionId        : 126

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestHeadersStop]: [System.Net.Http] Event: RequestHeadersStop

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestContentStart]: [System.Net.Http] Event: RequestContentStart

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestContentStop]: [System.Net.Http] Event: RequestContentStop
  contentLength       : 210

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-ConnectionClosed]: [System.Net.Http] Event: ConnectionClosed
  versionMajor        : 1
  versionMinor        : 1
  connectionId        : 126

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestFailed]: [System.Net.Http] Event: RequestFailed
  exceptionMessage    : An error occurred while sending the request.

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestFailedDetailed]: [System.Net.Http] Event: RequestFailedDetailed
  exception           : System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)

2025-10-02 08:42:27 -04 - [Verbose-Duplicati.Library.Main.NetworkTrafficLogger.Http-RequestStop]: [System.Net.Http] Event: RequestStop
  statusCode          : -1

2025-10-02 08:42:27 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Backend.Handler-RemoteOperationDelete]: RemoteOperationDelete took 0:00:00:00.377
2025-10-02 08:42:27 -04 - [Retry-Duplicati.Library.Main.Backend.Handler-RetryDelete]: Operation Delete with file duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes attempt 1 of 0 failed with message: An error occurred while sending the request.
System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetResponseAsync(HttpRequestMessage req, HttpCompletionOption httpCompletionOption, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetResponseAsync(HttpRequestMessage req, HttpCompletionOption httpCompletionOption, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.ReadJsonResponseAsync[T](HttpRequestMessage req, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetJsonDataAsync[T](String url, CancellationToken cancellationToken, Action`1 setup)
   at Duplicati.Library.JsonWebHelperHttpClient.PostAndGetJsonDataAsync[T](String url, Object item, CancellationToken cancellationToken)
   at Duplicati.Library.Backend.Backblaze.B2.<>c__DisplayClass34_1.<<DeleteAsync>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at Duplicati.Library.Utility.Utility.WithTimeout[T](TimeSpan timeout, CancellationToken token, Func`2 func)
   at Duplicati.Library.Backend.Backblaze.B2.DeleteAsync(String remotename, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.DeleteOperation.ExecuteAsync(IBackend backend, CancellationToken cancelToken)
   at Duplicati.Library.Main.Backend.BackendManager.DeleteOperation.ExecuteAsync(IBackend backend, CancellationToken cancelToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.Execute[TResult](PendingOperation`1 op, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.Execute(PendingOperationBase op, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.Handler.ExecuteWithRetry(PendingOperationBase op, CancellationToken cancellationToken)
2025-10-02 08:42:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Failed: duplicati-bd8f02cd24a9f4793b93bff20f0d6a3a4.dblock.zip.aes (9.92 MiB)
2025-10-02 08:42:27 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Backend.Handler-RemoteOperationDelete]: Starting - RemoteOperationDelete
2025-10-02 08:42:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i95857d83f8594464a0b02dbb7c00c37d.dindex.zip.aes (5.34 KiB)

Bottom line is the dindex file for the dblock, which deleted OK. The dblock didn’t delete.

In RemoteOperation table, Data column for dblock delete is not usual NULL but a stack

System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.<SendCoreAsync>g__Core|4_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetResponseAsync(HttpRequestMessage req, HttpCompletionOption httpCompletionOption, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetResponseAsync(HttpRequestMessage req, HttpCompletionOption httpCompletionOption, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.ReadJsonResponseAsync[T](HttpRequestMessage req, CancellationToken cancellationToken)
   at Duplicati.Library.JsonWebHelperHttpClient.GetJsonDataAsync[T](String url, CancellationToken cancellationToken, Action`1 setup)
   at Duplicati.Library.JsonWebHelperHttpClient.PostAndGetJsonDataAsync[T](String url, Object item, CancellationToken cancellationToken)
   at Duplicati.Library.Backend.Backblaze.B2.<>c__DisplayClass34_1.<<DeleteAsync>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at Duplicati.Library.Utility.Utility.WithTimeout[T](TimeSpan timeout, CancellationToken token, Func`2 func)
   at Duplicati.Library.Backend.Backblaze.B2.DeleteAsync(String remotename, CancellationToken cancellationToken)
   at Duplicati.Library.Main.Backend.BackendManager.DeleteOperation.ExecuteAsync(IBackend backend, CancellationToken cancelToken)

EDIT 1:

Isn’t it a bug if a failed B2 delete is considered a success? I wonder if a failed B2 put can be too, which may explain the missing files that’s been seen (grasping at straws, I know).

EDIT 2:

I didn’t actually go look at how my puts went that morning, but network was VERY flaky, which certainly contributed to a bunch of mess. I’m reporting only the bad result that I got.

ts678 · October 4, 2025, 2:34pm

github.com/duplicati/duplicati

Canary job log not produced upon error since 2.1.0.121

opened 01:56PM - 04 Oct 25 UTC

ts678

- [x] I have searched open and closed issues for duplicates. - [x] I have se…arched the [forum](https://forum.duplicati.com) for related topics. --- ## Environment info - **Duplicati version**: Canary series 2.1.0.121-2.1.1.104 so far. Fortunately Beta seems OK. - **Operating system**: Windows 10 - **Backend**: various ## Description The job log seems to be suppressed if any error occurs. In ngax, one gets the red popup, but no job log to see. Generally the server log gets the error, but is not what the GUI button shows by default, and is not well known. This will cause user frustration and extra support work. When error occurs, log info should exist, with easy path. ``` version popup job server 2.1.0.120 Y Y Y 2.1.0.121 N N Y 2.1.0.122 Y N Y 2.1.1.103 Y N Y 10/03/2025 releases: 2.1.1.104 Y N Y 2.1.2.1 Beta Y Y Y No such host is known. (api.backblazeb2.com:443) No such host is known. (duplicati-oauth-handler.appspot.com:443) ``` Earlier Canary releases generally had a B2 backup sitting in them, so I turned off Wi-Fi for "Verify files". 2.1.1.104 is so new that I made a backup of a file (to set DB), then switched to B2 with a dummy login. I was wishing I had set `number-of-retries=0` like on my usual, because it took awhile to actually fail... 2.1.2.1 Beta was a past-due OneDrive scheduled backup that ran, so I didn't work much to provoke it. I also tested 2.1.1.104 using an invalid source path, changing to a bad drive letter. Usual log loss held. Warning was also tested there (a little bit) using `no-local-blocks`, which is deprecated. Got a job log. Initial forum report [here](https://forum.duplicati.com/t/release-2-1-1-103-canary-2025-09-25/21349/15?u=ts678) had more comment, less detail, and some discussion. The general report was: > Compared to 2.1.2.0_beta_2025-08-20, job logs tend to go missing instead of giving error ## Steps to reproduce 1. Do something that gets an error - **Actual result**: No job log - **Expected result**: Job log ## Screenshots ## Debug log

filed with version tests, including yesterday’s two new releases. Fortunately Beta is OK.

Failed delete being treated as success still seems wrong, and I made test steps for that:

Make OneDrive job with upload-unchanged-backups=True, number-of-retries=0, no-backend-verification=True, and a short text file for Source.
Run backup twice
Disconnect Internet
Do Commandline delete of version=0

   Deleting file duplicati-20251004T141438Z.dlist.zip  (663 bytes) ... 
 These filesets were deleted: 
 0: 10/4/2025 10:14:38 AM 
 Return code: 0

Yet database seems like it’s setting up for a failure in two hours after DeleteGraceTime:

Remotevolume table. Why Deleted when delete failed? Would Deleting State do better?

ID	OperationID	Name	Type	Size	Hash	State	VerificationCount	DeleteGraceTime
1	1	duplicati-20251004T141251Z.dlist.zip	Files	663	SMwzOF7YOrsgAuBqKT/EuNBO8/vQ5Ch/cVPDAvXfEWM=	Verified	1	0
2	1	duplicati-b0007ad6e91b74901a7372118a75080ee.dblock.zip	Blocks	717	KqLu1x04NmUIPv5MpiaC8obPO8Uc7Iys1ymOso8a8O4=	Verified	2	0
3	1	duplicati-i075ea55c86464fc09d0c3db867a640af.dindex.zip	Index	617	XzyH0hcQtqMHj4cGKUcXyhwVIN0kTTRgMqdnoRHHGCM=	Verified	2	0
4	5	duplicati-20251004T141438Z.dlist.zip	Files	-1		Deleted	1	1759594659

RemoteOperation table for the delete:

ID	OperationID	Timestamp	Operation	Path	Data
15	5	1759587459	delete	duplicati-20251004T141438Z.dlist.zip	System.TimeoutException: The operation has timed out. at Duplicati.Library.Utility.Utility.WithTimeout[T](TimeSpan timeout, CancellationToken token, Func`2 func) at Duplicati.Library.Backend.MicrosoftGraphBackend.DeleteAsync(String remotename, CancellationToken cancelToken) at Duplicati.Library.Main.Backend.BackendManager.DeleteOperation.ExecuteAsync(IBackend backend, CancellationToken cancelToken)

EDIT 1:

I forgot to say that test is a yesterday’s release, not 2.1.1.103. Turns out it’s 2.0.2.1 Beta.

EDIT 2:

DeleteGraceTime was 10:17:39 AM. Turned off no-backend-verification to run Verify files

2025-10-04 12:15:06 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-20251004T141438Z.dlist.zip

2025-10-04 12:20:04 -04 - [Information-Duplicati.Library.Main.Operation.FilelistProcessor-IgnoreRemoteDeletedFile]: Ignoring remote file listed as Deleted: duplicati-20251004T141438Z.dlist.zip

Above run deleted the row, but next one found the error. Presumably order of operations.

2025-10-04 12:21:11 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-ExtraUnknownFile]: Extra unknown file: duplicati-20251004T141438Z.dlist.zip
2025-10-04 12:21:11 -04 - [Error-Duplicati.Library.Main.Operation.FilelistProcessor-ExtraRemoteFiles]: Found 1 remote files that are not recorded in local storage. This can be caused by having two backups sharing a destination folder which is not supported. It can also be caused by restoring an old database. If you are certain that only one backup uses the folder and you have the most updated version of the database, you can use repair to delete the unknown files.

As more evidence that the missing log issue is fortunately not in 2.0.2.1 Beta release, logs:

EDIT 3:

Testing only up through the delete-without-network, then looking at Remotevolume table,

2.0.2.0 Beta shows the same issue
2.1.0.5 is OK (maybe). Database is

Remotevolume is

ID	OperationID	Name	Type	Size	Hash	State	VerificationCount	DeleteGraceTime
4	4	duplicati-20251004T165301Z.dlist.zip	Files	664		Deleting	0	0

RemoteOperation has no entry for the delete. Try looking for a delete in the profiling log:

2025-10-04 12:54:50 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.BackendManager-RemoteOperationDelete]: Starting - RemoteOperationDelete
2025-10-04 12:54:50 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-20251004T165301Z.dlist.zip (664 bytes)
2025-10-04 12:55:22 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.BackendManager-RemoteOperationDelete]: RemoteOperationDelete took 0:00:00:31.043
2025-10-04 12:55:22 -04 - [Retry-Duplicati.Library.Main.BackendManager-RetryDelete]: Operation Delete with file duplicati-20251004T165301Z.dlist.zip attempt 1 of 0 failed with message: Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2
Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2
 ---> System.Net.WebException: No such host is known. (duplicati-oauth-handler.appspot.com:443)
 ---> System.Net.Http.HttpRequestException: No such host is known. (duplicati-oauth-handler.appspot.com:443)
 ---> System.Net.Sockets.SocketException (11001): No such host is known.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.HttpWebRequest.<>c__DisplayClass219_0.<<CreateHttpClient>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at System.Net.HttpWebRequest.SendRequest(Boolean async)
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](HttpWebRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   at Duplicati.Library.OAuthHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Duplicati.Library.Backend.MicrosoftGraphBackend.DeleteAsync(String remotename, CancellationToken cancelToken)
   at Duplicati.Library.Main.BackendManager.DoDeleteAsync(FileEntryItem item, CancellationToken cancellationToken)
   at Duplicati.Library.Utility.Utility.Await(Task task)
   at Duplicati.Library.Main.BackendManager.ThreadRun()
2025-10-04 12:55:22 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Failed: duplicati-20251004T165301Z.dlist.zip (664 bytes)
2025-10-04 12:55:22 -04 - [Information-Duplicati.Library.Main.BackendManager-DeleteFileFailed]: Failed to delete file duplicati-20251004T165301Z.dlist.zip, testing if file exists
2025-10-04 12:55:22 -04 - [Warning-Duplicati.Library.Main.BackendManager-DeleteFileFailure]: Failed to recover from error deleting file duplicati-20251004T165301Z.dlist.zip
System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Main.BackendManager.ThreadRun()
2025-10-04 12:55:22 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: Starting - ExecuteNonQuery: PRAGMA optimize
2025-10-04 12:55:22 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-10-04 12:55:22 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Controller-RunDelete]: Running Delete took 0:00:00:31.475
2025-10-04 12:55:22 -04 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Delete has failed with error: Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2
Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2
 ---> System.Net.WebException: No such host is known. (duplicati-oauth-handler.appspot.com:443)
 ---> System.Net.Http.HttpRequestException: No such host is known. (duplicati-oauth-handler.appspot.com:443)
 ---> System.Net.Sockets.SocketException (11001): No such host is known.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|285_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.HttpWebRequest.<>c__DisplayClass219_0.<<CreateHttpClient>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at System.Net.HttpWebRequest.SendRequest(Boolean async)
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](HttpWebRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.Main.BackendManager.WaitForComplete(LocalDatabase db, IDbTransaction transation)
   at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager)
   at Duplicati.Library.Main.Operation.DeleteHandler.Run()
   at Duplicati.Library.Main.Controller.<Delete>b__20_0(DeleteResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

Commandline output shows the idea, plus an extra error that I’m hoping isn’t interfering:

  Deleting file duplicati-20251004T165301Z.dlist.zip  (664 bytes) ...
Failed to recover from error deleting file duplicati-20251004T165301Z.dlist.zip => Object reference not set to an instance of an object.
The operation Delete has failed with error: Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2 => Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2

ErrorID: OAuthLoginError
Failed to authorize using the OAuth service: No such host is known. (duplicati-oauth-handler.appspot.com:443). If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=onedrivev2
Return code: 100

Remotevolume in 2.1.0.5 looks how I was thinking newer Beta should look, Deleting.

Conclusion so far is that this looks like a regression, but feedback about it is welcomed.

EDIT 4:

While waiting discussion somewhere, I filed an issue because this is impacting the Beta.

github.com/duplicati/duplicati

remote delete failure sets bad DB state, gets "not recorded in local storage" hours later

opened 05:35PM - 04 Oct 25 UTC

ts678

- [x] I have searched open and closed issues for duplicates. - [x] I have se…arched the [forum](https://forum.duplicati.com) for related topics. --- ## Environment info - **Duplicati version**: recent Beta and Canary - **Operating system**: Windows 10 - **Backend**: various ## Description "remote files that are not recorded in local storage." error occurs unexpectedly two or more hours after destination `delete` fails. [Release: 2.1.1.103 (Canary) 2025-09-25](https://forum.duplicati.com/t/release-2-1-1-103-canary-2025-09-25/21349/19?u=ts678) is analysis (after hitting this on a bad network day), steps to reproduce it, and version test. Unfortunately, both 2.0.2.0 Beta and 2.0.2.1 Beta have the problem. 2.1.0.5 Stable looks like it doesn't, so this seems a regression.

Sami_Lehtinen · October 7, 2025, 11:39am

Sorry, didn’t check if this is reported and or fixed already. It seems that compact fails with internal locking issue. This happens repeatedly. Just with ordinary local compaction task, which should have no problems whatsoever to run.

Version:  - 2.1.1.103_canary_2025-09-25

The operation Compact has failed => SQLite Error 5: ‘database is locked’.

Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 5: ‘database is locked’.at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)at Microsoft.Data.Sqlite.SqliteDataReader.NextResult()at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteScalarInt64Async(SqliteCommand self, Boolean writeLog, String cmd, Dictionary2 values, Int64 defaultvalue, CancellationToken cancellationToken)    at Duplicati.Library.Main.Database.LocalDatabase.CreateLocalDatabaseAsync(SqliteConnection connection, String operation, LocalDatabase dbnew, CancellationToken token)    at Duplicati.Library.Main.Database.LocalDatabase.CreateLocalDatabaseAsync(SqliteConnection connection, String operation, LocalDatabase dbnew, CancellationToken token)    at Duplicati.Library.Main.Database.LocalDatabase.CreateLocalDatabaseAsync(String path, String operation, Boolean shouldclose, LocalDatabase db, CancellationToken token)    at Duplicati.Library.Main.Operation.VacuumHandler.RunAsync()    at Duplicati.Library.Main.Operation.CompactHandler.RunAsync(IBackendManager backendManager)    at Duplicati.Library.Main.Operation.CompactHandler.RunAsync(IBackendManager backendManager)    at Duplicati.Library.Utility.Utility.Await(Task task)    at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Func3 method)at Duplicati.Library.Main.Controller.RunAction[T](T result, Func3 method)    at Duplicati.Library.Main.Controller.Compact()    at Duplicati.CommandLine.Commands.Compact(TextWriter outwriter, Action1 setup, List1 args, Dictionary2 options, IFilter filter)at Duplicati.CommandLine.Program.ParseCommandLine(TextWriter outwriter, Action1 setup, Boolean& verboseErrors, String[] args)    at Duplicati.CommandLine.Program.RunCommandLine(TextWriter outwriter, TextWriter errwriter, Action1 setup, String args)

I’ll add two extra details:
–threshold=0 - And it failed interestingly exactly when it should have been ready.

Could have been also one of those quirks where duplicati leaves processes / locks lingering.

I’ll get back to this if it happens repeatedly in future.