Release: 2.1.0.119 (Canary) 2025-05-29

kenkendk · May 29, 2025, 6:20am

2.1.0.119_canary_2025-05-29

This release is a canary release intended to be used for testing.

The intention is to tap this release as an experimental release if no significant issues are uncovered.

Major changes in this version

Index file corruption fixed

This update fixes a long-standing issue with index files being incomplete over time.
The effect of this is that the recreate database step takes much longer than expected.
With this update the index files no longer loose information during the compact step.

To remedy existing data, the test method can now be run and only check index files (--full-remote-verification=indexesonly).
By default, any defective index files that are found will be replaced with correct versions, so simply running the test command with a large sample size will fix the backup.

Suppress warnings

A new option is added that can change warning messages to information messages, so the warnings no longer generate notifications.
The file access errors are now better grouped so it shows if the error is a permission error, a locked file error and others.
With the suppress feature it is possible to suppress one or more categories of these warnings.

Better backend Test

The test feature has been updated to now check for write permissions, instead of just listing the remote folder.
For ngclient, the Test check now uses this extended check to inform the user if there are files in the destination folder.
This shows a warning if creating a new backup pointing to a folder with existing data, and a warning if attempting to restore from a folder with no files.
If the write part fails for a test during the restore flow, this is not treated as an error in ngclient.

Auto VSS on Windows

If the user has Administrator privileges (SeBackupPrivilege) VSS is now set to Auto by default.

Updated backends

Updated the backends: Jottacloud, Google Drive, Google Cloud Storage, Dropbox, and OpenStack.
Removed the Sia backend as the current implementation is not compatible with the pending hard-fork.

No throttle for the file backend

The file backend is now excempt (by default) from throttle settings via --throttle-disabled-backends.
Individual backups can set --throttle-disable to opt-out of throttling.

Disabling reported options on Webserver

The ngclient is using the reported options to figure out what features can be enabled.
This can now be configured with --webservice-disable-api-extensions so websocket usage can be disabled, among others.

ngclient updates

A number of fixes and additions are made to ngclient, including a new grid-view for users with many backups.

Database version upgrade to v17

The local database is again upgraded with an index changes performance.
The bundled Duplicati.CommandLine.DatabaseTool.exe / duplicati-database-tool can downgrade databases.
Since the change is only index addition, there is no data loss on downgrades.

Detailed list of changes:

Fixed issue with empty filesets being created
Fixed issue with transactions being double comitted
Fixed a case where a purge could loose a version
Added option to register for remote control on TrayIcon startup
Fixed an issue with purge-broken-files
Fixed list-broken-files and purge-broken-files not reading database config
Removed multiple unused libraries
Fixed dbtool checks on Windows
Added an extra check for the secret provider to ensure the modules behave
Fixed a crash on serialization of result message
Added an index to blockset entry
Fixed an issue with compact not writing blocklists into index files
Prevent overwriting config files with RPM install
Add classification of file access errors
Added feature to suppress warnings
Fixed CIFS backend not parsing the writebuffer option
More robust flushing of database messages on stops
VSS is set to Auto on Windows, if the user has permissions to make snapshots
Fixed incorrect text on compression level
Removed mentions of removed backends
Preserve space during compact by deleting files early, instead of at the end of the compact
Reduced memory usage in SyncTool
Fixed an issue where timestamps could drift and cause validation errors
Updated translations, thanks to all translators
Updated the Test method to also test for write-permissions (all backends)
Removed the Sia backend due to incompatibility with hard-fork
Updated GoogleDrive, GCS, and Dropbox to use HttpClient
Updated Jottacloud to use HttpClient
Updated OpenStack to use HttpClient
Added simpler format for SSH key
Removed old libraries and files that caused build warnings
Default disable throttle for the file backend (configurable)
Removed global options to set various unsupported options, in favor of per-backend options
Improved error message if the backend is not a supported one
Added support for repairing broken remote index files
Improved response time for the Stop command
Added a number of messages to websockets to automatically push updates to the browser
Added support for toggling reported server features off
Fixed a deadlock on restore when transfers failed

Ngclient changes:

Fixed avoiding colon for default port on destination url
Fixed buttons on database page to update
Fixed an issue where commandline would have extra options
Showing the name of the backup being edited
Fixed showing the correct relative time
Sliders now show correctly when editing
Added Server State to About page
Added a grid-based view of the backup list
Showing the backend type in the backup list
Ask to resume server if paused when activating command
Improved throttle dialog
Updated translations, thanks to all translators
Added pluralization and other minor fixes to the backup view
Improved color scheme
Added drop-area for SSH keyfile
Added warning on leading slash for paths in destination
Advanced options show a warning if they are overriding a global option when editing
Fixed tooltips
Minor fixes to the SSH page
Notifications are slightly more visible now
Added page to do advanced job delete
Added warnings when leaving the backup page without saving
Removed HttpOptions area on destination
Support for custom OAuth url
Added deprecation information to advanced options
Fixed advanced options not behaving the same in ngax and ngclient
Added a stop / abort button to tasks
Don’t show warnings when checking if files exist

Folgore101 · May 29, 2025, 8:41am

Hi @kenkendk, I think there is some problem with the implementation of the translations, for example the Italian translation of ngclient should be complete (apart from the last 27 strings released yesterday) but the interface remains in English.
Even putting Dutch (nl-NL) which is the most complete, only the 27 strings released yesterday are missing, the new interface remains in English, can you check it out?
Thank you.

ts678 · May 29, 2025, 2:10pm

Release: 2.1.0.118 (Canary) 2025-05-12 has many ngclient issues without response.
I haven’t gone through them all yet to see if any were fixed, but first one I tested isn’t.
I was hoping to get a bit of discussion. For more clearcut bugs, I’ve been going direct

https://github.com/duplicati/ngclient/issues

ts678 · May 29, 2025, 5:47pm

Restore request never finishes (or so the UI reports, but it might not be a good report):

Technically it says “The operation Restore has completed…” but then it goes on to hang.

2025-05-29 13:35:57 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Restore has completed
2025-05-29 13:35:58 -04 - [Error-Duplicati.Server.Program-UnobservedTaskException]: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The channel "" is retired)
 ---> CoCoL.RetiredException: The channel "" is retired
   at CoCoL.Channel`1.ReadAsync(ITwoPhaseOffer offer)
   --- End of inner exception stack trace ---
2025-05-29 13:37:00 -04 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option ----compression-module is not supported and will be ignored
2025-05-29 13:37:00 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation ListFilesets has started
2025-05-29 13:37:00 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Controller-RunListFilesets]: Starting - Running ListFilesets
2025-05-29 13:37:00 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjM3RjJCRTZDNkJCNzNGRjciLCJuYmYiOjE3NDg1NDAyMjAsImV4cCI6MTc0ODU0MTEyMCwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.lULOkLkszq3Z3tYlxeQRwpYJ-EKiSSaCpmmVWzrvqek
2025-05-29 13:37:00 -04 - [Verbose-Duplicati.Library.SQLiteHelper.SQLiteLoader-CustomSQLiteOption]: Setting custom SQLite option 'cache_size=-333824'.
2025-05-29 13:37:00 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteScalarInt64]: Starting - ExecuteScalarInt64: INSERT INTO "Operation" ("Description", "Timestamp") VALUES (@Description, @Timestamp); SELECT last_insert_rowid();
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteScalarInt64]: ExecuteScalarInt64: INSERT INTO "Operation" ("Description", "Timestamp") VALUES (@Description, @Timestamp); SELECT last_insert_rowid(); took 0:00:00:00.326
2025-05-29 13:37:01 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: Starting - ExecuteNonQuery: PRAGMA optimize
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-05-29 13:37:01 -04 - [Verbose-Duplicati.Library.SQLiteHelper.SQLiteLoader-CustomSQLiteOption]: Setting custom SQLite option 'cache_size=-333824'.
2025-05-29 13:37:01 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteScalarInt64]: Starting - ExecuteScalarInt64: INSERT INTO "Operation" ("Description", "Timestamp") VALUES (@Description, @Timestamp); SELECT last_insert_rowid();
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteScalarInt64]: ExecuteScalarInt64: INSERT INTO "Operation" ("Description", "Timestamp") VALUES (@Description, @Timestamp); SELECT last_insert_rowid(); took 0:00:00:00.169
2025-05-29 13:37:01 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: Starting - ExecuteNonQuery: PRAGMA optimize
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Controller-RunListFilesets]: Running ListFilesets took 0:00:00:00.575
2025-05-29 13:37:01 -04 - [Verbose-Duplicati.Library.SQLiteHelper.SQLiteLoader-CustomSQLiteOption]: Setting custom SQLite option 'cache_size=-333824'.
2025-05-29 13:37:01 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteReader]: Starting - ExecuteReader: SELECT "ID", "Timestamp" FROM "Operation" ORDER BY "Timestamp" DESC LIMIT 1
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteReader]: ExecuteReader: SELECT "ID", "Timestamp" FROM "Operation" ORDER BY "Timestamp" DESC LIMIT 1 took 0:00:00:00.000
2025-05-29 13:37:01 -04 - [Profiling-Timer.Begin-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: Starting - ExecuteNonQuery: PRAGMA optimize
2025-05-29 13:37:01 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-05-29 13:37:01 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation ListFilesets has completed

EDIT 1:

Duplicating the tab gets a slightly different log view, but it still thinks it’s stuck.
System info (have to use ngax) doesn’t look quite so stuck. Maybe GUI bug?

Server state properties
lastEventId : 52
lastDataUpdateId : 24
lastNotificationUpdateId : 12
estimatedPauseEnd : "0001-01-01T00:00:00.000Z"
activeTask :
programState : Running
lastErrorMessage :
connectionState : connected
connectionAttemptTimer : 0
failedConnectionAttempts : 0
failedAuthAttempts : 0
lastPgEvent :
updaterState : Waiting
updateDownloadLink :
updatedVersion :
updateDownloadProgress : 0
proposedSchedule : []
schedulerQueueIds : []
orderBy : id
pauseTimeRemain : 0

EDIT 2:

If I ignore Restore claim that it’s “Running” and go to the Home screen, things seem idle.
My file did restore (but not its timestamp, as reported on 2.1.0.118 announce). Checking equivalent profiling log for 2.1.0.118 didn’t see the UnobservedTaskException seen here.

EDIT 3:

I don’t know if this is consistent, but I used ngax “Edit as text” to correct four-dash option.
Now I’m not getting that exception, but I’m still getting a stuck “Running”. End is now like:

2025-05-29 14:14:08 -04 - [Profiling-Timer.Finished-Duplicati.Library.Main.Database.ExtensionMethods-ExecuteNonQuery]: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
2025-05-29 14:14:08 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Restore has completed

Taomyn · May 30, 2025, 8:54am

Have updated my various machines and nothing to report on that so far. I can confirm that the RPM installer is no longer resetting my service config file

Currently I’m testing the index file checker via the command-line against one of my Windows servers - on the main local backup it’s found many issues and still waiting to see the result - it’s taking some time.

Regarding the job command-line on the GUI, but are there any plans to make this work better? For example, when you change it to “test” it leaves all the commands/parameters from the backup most of which will cause the command to fail if you forget to edit them (losing what you did), and even when you remember, trying to figure out which ones are needed to remove the rest is a real pain. Can’t the GUI tidy up the commands/parameter to leave only the ones relevant to the command we’re trying to run? Hope that makes sense.

Taomyn · May 30, 2025, 12:34pm

I’ve run the “test” command numerous times against one of my local backups, and it’s repaired some indexes which is great, but I’m wondering what I need to do about this:

  "Errors": [
    "2025-05-30 14:19:52 +02 - [Error-Duplicati.Library.Main.Operation.TestHandler-Test results]: Verified 2358 remote files with 59 problem(s)"

The command line is showing the dblocks with issues so I can use that to target a fix, but I don’t remember what that was. If there’s a way to avoid renaming the files that would be a huge benefit as Wasabi S3 has a 3 month deletion retention that I get charged for.

The test command is also showing up the warnings about the LZMA fallback, but this is disrupting all my backups now. Is there any chance of a way to suppress these particular warnings but keep them logged for monitoring? I really don’t want to enable SharpCompress and would prefer to keep to Duplicati defaults to avoid this happening again - I’m hoping over time these files will get cleaned up when they become empty.

ts678 · May 30, 2025, 7:53pm

Not sure what below from profiling log is about, but 2.1.0.118 wasn’t logging such stuff:

2025-05-29 21:38:12 -04 - [Warning-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketInvalidToken]: WebSocket connection with invalid token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjM3RjJCRTZDNkJCNzNGRjciLCJuYmYiOjE3NDg1NDE2NTIsImV4cCI6MTc0ODU0MjU1MiwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.HMlA0CsOI1gCKi5DGY1Akj7UE8DCg_gY3sjpwjUIHYs
Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException: IDX10223: Lifetime validation failed. The token is expired. ValidTo (UTC): '5/29/2025 6:15:52 PM', Current time (UTC): '5/30/2025 1:38:12 AM'.
   at Microsoft.IdentityModel.Tokens.Validators.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, JwtSecurityToken jwtToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateTokenPayload(JwtSecurityToken jwtToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateJWS(String token, TokenValidationParameters validationParameters, BaseConfiguration currentConfiguration, SecurityToken& signatureValidatedToken, ExceptionDispatchInfo& exceptionThrown)
--- End of stack trace from previous location ---
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, JwtSecurityToken outerToken, TokenValidationParameters validationParameters, SecurityToken& signatureValidatedToken)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, TokenValidationParameters validationParameters, SecurityToken& validatedToken)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ParseAndValidateToken(String token, TokenType tokenType)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ReadAccessToken(String token)
   at Duplicati.WebserverCore.Notifications.WebsocketAuthenticator.HandleClientMessage(WebSocket socket, String messagestr)

Looking over Websocket message history, it switched its style of message at this point:

2025-05-29 17:47:41 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjM3RjJCRTZDNkJCNzNGRjciLCJuYmYiOjE3NDg1NTUyNjEsImV4cCI6MTc0ODU1NjE2MSwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.PzAyh6LuMHpR6EiW2s3M47jjxtp46QJqC4yAarBFohQ
2025-05-29 17:48:21 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjM3RjJCRTZDNkJCNzNGRjciLCJuYmYiOjE3NDg1NTUzMDEsImV4cCI6MTc0ODU1NjIwMSwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.xD62JefkYldGL9eZjFZyTx583Llj5HfHP4oIk6dbCCU
2025-05-29 18:41:59 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-05-29 18:42:18 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-05-29 18:47:38 -04 - [Information-Duplicati.Server.Program-ServerStopping]: Server is stopping, tearing down handlers
2025-05-29 18:47:48 -04 - [Information-Duplicati.Server.Program-ServerStopped]: Server has stopped
2025-05-29 21:06:34 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-05-29 21:06:36 -04 - [Information-Duplicati.Server.WebServerLoader-ServerListening]: Server has started and is listening on 0.0.0.0, port 8119
2025-05-29 21:06:36 -04 - [Information-Duplicati.Server.Program-ServerStarted]: Server has started and is listening on *, port 8119
2025-05-29 21:07:27 -04 - [Warning-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketInvalidToken]: WebSocket connection with invalid token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjM3RjJCRTZDNkJCNzNGRjciLCJuYmYiOjE3NDg1NDE2NTIsImV4cCI6MTc0ODU0MjU1MiwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.HMlA0CsOI1gCKi5DGY1Akj7UE8DCg_gY3sjpwjUIHYs

Taomyn · June 2, 2025, 9:14am

Tried to see if I could repair one of the backups that keeps throwing an error and getting a bit stuck. I ran from the command-line a list-broken-files and it showed 1000 lines of output. So I ran a purge-broken-files with the --dry-run to start with, but it’s not doing anything and just sits there.

This is the command I’m running, from Powershell as I like to keep the commands in history for later use: & 'C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe' purge-broken-files file://D:\LISA\DUPLICATI\LOCAL\ --asynchronous-upload-folder=T:\Duplicati\ --tempdir=T:\Duplicati\ --dbpath=C:\ProgramData\Duplicati\ICXFGPFETH.sqlite --dblock-size=100MB --passphrase=<redacted> --dry-run basically the same as I did for the list-broken-files

Watching the process in Resource Monitor this is all that shows after a short while of showing many other files in use:

I can’t see any other Duplicati processes using any files, although the main service is still running, but no backups or other jobs are running.

I waited for about an 20min and killed it before just trying again - still the same. I then tried the same command in a command-prompt and that’s no different.

Any suggestions?

Taomyn · June 2, 2025, 1:01pm

Thought I would try the GUI and it’s doing the same: current time is 15:00 and it’s been like this since 13:55:

It does show something has happened as those messages never came up anywhere else when I used the command line

FreaQ · June 3, 2025, 6:50am

Does this fix this issue?

ts678 · June 3, 2025, 2:04pm

The SecurityTokenExpiredException mentioned above seems to repro pretty easily.
The result, in addition to warnings, seems to be that automatic reconnect doesn’t work:

I can get this by starting Duplicati in the evening, starting Edge dev tools, doing a hard refresh to make sure the Websocket is setup and Edge is watching it, and sleeping PC.

Around 3 A.M, Windows wakes for an update check, and Duplicati starts flailing around:

Process started at 9:32:39

2025-06-02 21:32:58 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-06-02 21:33:01 -04 - [Information-Duplicati.Server.WebServerLoader-ServerListening]: Server has started and is listening on 0.0.0.0, port 8119
2025-06-02 21:33:01 -04 - [Information-Duplicati.Server.Program-ServerStarted]: Server has started and is listening on *, port 8119
2025-06-02 21:34:21 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-06-02 21:34:26 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjI0MThENzlGOTlEMDhENTQiLCJuYmYiOjE3NDg5MTQ0NjUsImV4cCI6MTc0ODkxNTM2NSwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.A7eZPbPY2IOrJw6EDG0hce74hLZzizNAbKZPiYjvjFU
2025-06-02 21:35:01 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjI0MThENzlGOTlEMDhENTQiLCJuYmYiOjE3NDg5MTQ1MDAsImV4cCI6MTc0ODkxNTQwMCwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.i6h8p_P8YtnMyKrJreBJGixx30Lz18FSyuT4oyanfl4
2025-06-03 02:59:50 -04 - [Warning-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketInvalidToken]: WebSocket connection with invalid token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjI0MThENzlGOTlEMDhENTQiLCJuYmYiOjE3NDg5MTQ1MDAsImV4cCI6MTc0ODkxNTQwMCwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.i6h8p_P8YtnMyKrJreBJGixx30Lz18FSyuT4oyanfl4
Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException: IDX10223: Lifetime validation failed. The token is expired. ValidTo (UTC): '6/3/2025 1:50:00 AM', Current time (UTC): '6/3/2025 6:59:50 AM'.
   at Microsoft.IdentityModel.Tokens.Validators.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, JwtSecurityToken jwtToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateTokenPayload(JwtSecurityToken jwtToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateJWS(String token, TokenValidationParameters validationParameters, BaseConfiguration currentConfiguration, SecurityToken& signatureValidatedToken, ExceptionDispatchInfo& exceptionThrown)
--- End of stack trace from previous location ---
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, JwtSecurityToken outerToken, TokenValidationParameters validationParameters, SecurityToken& signatureValidatedToken)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, TokenValidationParameters validationParameters, SecurityToken& validatedToken)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ParseAndValidateToken(String token, TokenType tokenType)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ReadAccessToken(String token)
   at Duplicati.WebserverCore.Notifications.WebsocketAuthenticator.HandleClientMessage(WebSocket socket, String messagestr)

Wireshark:

The repro is possibly more complex than need be. Dev tools didn’t add much, and I’m not certain that the hard refresh was necessary. I used ngclient, but I don’t know if it mattered.

EDIT 1:

A very simplified test with theme.html bring up ngax didn’t have the same no-connect/loop.

doesn’t look great, but ngax seemed to wind up as expected in the end, allowing my use.

EDIT 2:

and ngclient continues to have trouble, so I’m tending to say this is more an ngclient bug.

Here’s a TrayIcon restart:

2025-06-04 13:11:58 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjBENUIwQjlBMUQxNzNDNzgiLCJuYmYiOjE3NDkwNTcxMTcsImV4cCI6MTc0OTA1ODAxNywiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.g8vb1DCw8EkTqo6h796OBxeL6_3UriBVOGxXGrm3aZA
2025-06-04 13:15:15 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjBENUIwQjlBMUQxNzNDNzgiLCJuYmYiOjE3NDkwNTczMTUsImV4cCI6MTc0OTA1ODIxNSwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.KZilQg6qCevpmXzoopHjr_lb8nvCwOQls2Ldrhh4Qi4
2025-06-04 13:18:01 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjA0MkM4MTE5NTdBN0U5NUQiLCJuYmYiOjE3NDkwNTc0ODAsImV4cCI6MTc0OTA1ODM4MCwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.Qs0I-ILnZWpqeeKWRaB1hswfvqxdoy-pyS90Yr2w19s
2025-06-04 13:40:04 -04 - [Profiling-Duplicati.Library.Snapshots.Windows.HyperVUtility-WMISelect]: Using WMI provider \\localhost\root\virtualization\v2
2025-06-04 13:40:04 -04 - [Information-Duplicati.Library.Snapshots.Windows.MSSQLUtility-NoMSSQLInstance]: Cannot find any MS SQL Server instance. MS SQL Server is probably not installed.
2025-06-04 13:40:04 -04 - [Information-Duplicati.Library.Snapshots.Windows.HyperVUtility-NoHyperVFound]: Cannot open WMI provider \\localhost\root\virtualization\v2. Hyper-V is probably not installed.
2025-06-04 21:43:23 -04 - [Information-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketAuthenticated]: WebSocket connection authenticated with token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjI0MThENzlGOTlEMDhENTQiLCJuYmYiOjE3NDkwODc4MDMsImV4cCI6MTc0OTA4ODcwMywiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.VcMEVEF4fPDV5J5x4FSK9jaB3CiCRyWXOY9n1rh7E7o
2025-06-04 21:43:52 -04 - [Information-Duplicati.Server.Program-ServerStopping]: Server is stopping, tearing down handlers
2025-06-04 21:44:03 -04 - [Information-Duplicati.Server.Program-ServerStopped]: Server has stopped
2025-06-04 21:44:47 -04 - [Verbose-GetGpgProgramPath-gpg]: 
2025-06-04 21:44:49 -04 - [Information-Duplicati.Server.WebServerLoader-ServerListening]: Server has started and is listening on 0.0.0.0, port 8119
2025-06-04 21:44:51 -04 - [Information-Duplicati.Server.Program-ServerStarted]: Server has started and is listening on *, port 8119
2025-06-04 21:44:53 -04 - [Warning-Duplicati.WebserverCore.Notifications.WebsocketAuthenticator-WebsocketInvalidToken]: WebSocket connection with invalid token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0eXAiOiJBY2Nlc3NUb2tlbiIsInNpZCI6IndlYnNlcnZlciIsImZhbSI6IjBENUIwQjlBMUQxNzNDNzgiLCJuYmYiOjE3NDkwNTczMTUsImV4cCI6MTc0OTA1ODIxNSwiaXNzIjoiaHR0cHM6Ly9kdXBsaWNhdGkiLCJhdWQiOiJodHRwczovL2R1cGxpY2F0aSJ9.KZilQg6qCevpmXzoopHjr_lb8nvCwOQls2Ldrhh4Qi4
Microsoft.IdentityModel.Tokens.SecurityTokenExpiredException: IDX10223: Lifetime validation failed. The token is expired. ValidTo (UTC): '6/4/2025 5:30:15 PM', Current time (UTC): '6/5/2025 1:44:53 AM'.
   at Microsoft.IdentityModel.Tokens.Validators.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, SecurityToken securityToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateLifetime(Nullable`1 notBefore, Nullable`1 expires, JwtSecurityToken jwtToken, TokenValidationParameters validationParameters)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateTokenPayload(JwtSecurityToken jwtToken, TokenValidationParameters validationParameters, BaseConfiguration configuration)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateJWS(String token, TokenValidationParameters validationParameters, BaseConfiguration currentConfiguration, SecurityToken& signatureValidatedToken, ExceptionDispatchInfo& exceptionThrown)
--- End of stack trace from previous location ---
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, JwtSecurityToken outerToken, TokenValidationParameters validationParameters, SecurityToken& signatureValidatedToken)
   at System.IdentityModel.Tokens.Jwt.JwtSecurityTokenHandler.ValidateToken(String token, TokenValidationParameters validationParameters, SecurityToken& validatedToken)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ParseAndValidateToken(String token, TokenType tokenType)
   at Duplicati.WebserverCore.Middlewares.JWTTokenProvider.ReadAccessToken(String token)
   at Duplicati.WebserverCore.Notifications.WebsocketAuthenticator.HandleClientMessage(WebSocket socket, String messagestr)

EDIT 3:

I can get it with Server instead of TrayIcon too (which reduces the network noise level).

ollibraun · June 4, 2025, 5:46am

To remedy existing data, the test method can now be run and only check index files (--full-remote-verification=indexesonly ).
By default, any defective index files that are found will be replaced with correct versions, so simply running the test command with a large sample size will fix the backup.

Can this be done via the WebGUI? How can a complete backup be tested?

ts678 · June 4, 2025, 1:11pm

Citing old manual, as new is not finished. Which GUI are people using?

Using the Command line tools from within the Graphical User Interface

The TEST command

If “all” is specified, all files in the backup will be tested.

So that goes into Commandline arguments. If lucky, most prior options can be left alone.

The option defaults to true (which is unusual for a boolean), but doesn’t seem to be considered true unless explicitly supplied as true. This seems against expectations.

Issue repro:

Backup a file larger than 1 block. Default blocksize is 1 MB but is configurable.
Open the dindex file (note name), e.g. in Explorer, and delete the list folder.
Use Database screen to Recreate the database, otherwise edit will be noticed.
Do the fix steps, then check for new dindex with new name with a list folder.

Running commandline entry
Finished!

            
  Listing remote folder ...
  Downloading file duplicati-i1a622b549d784c5981003b459080e502.dindex.zip (702 bytes) ...
Found 1 faulty index files, use the option --replace-faulty-index-files to repair them
Verified 1 remote files with 1 problem(s)
duplicati-i1a622b549d784c5981003b459080e502.dindex.zip: 1 errors
	Missing: TE22wCHk2ZCMDmPgRmZrPOQY2tifA17OQOkzUAgwj8U=

Return code: 3

If I explicitly specify false, it fails. If I explicitly specify true or no value (Edit as text), it fixes.

Running commandline entry
Finished!

            
  Listing remote folder ...
  Downloading file duplicati-i1a622b549d784c5981003b459080e502.dindex.zip (702 bytes) ...
Found 1 faulty index files, repairing now
  Uploading file duplicati-i5f83e4341819406bb3797465a39b44f7.dindex.zip (940 bytes) ...
  Deleting file duplicati-i1a622b549d784c5981003b459080e502.dindex.zip  (702 bytes) ...
Verified 1 remote files with 1 problem(s)
duplicati-i1a622b549d784c5981003b459080e502.dindex.zip: 1 errors
	Missing: TE22wCHk2ZCMDmPgRmZrPOQY2tifA17OQOkzUAgwj8U=

Return code: 3

however the output is a bit confusing at first glance. It seems to end with failure, but that’s the old file which was found to have a problem, so was replaced (uploaded) then deleted.

Taomyn · June 5, 2025, 10:10am

I decided to try the TEST command against one of my troubled backups, and I’m stuck on how to get it to fix it. I ran it with the --replace-faulty-index-files but this is the result each time, which I assume is because these aren’t all index files:

Found 59 faulty index files, use the option --replace-faulty-index-files to repair them
Verified 2471 remote files with 59 problem(s)
duplicati-ifdb6a15cca9f4ff580698f51516839ad.dindex.zip.aes: 547 errors
        Missing: +AkicAM0wYd52pwuwQhEKNPmClAR9zabKJaIpfHevBM=
        Missing: +EQ2kcR2dodPBxnqMqLtB5DFK5OuOPgKktjEl9dZVLE=
        Missing: +JJ01xCo89u6y82A544fylWPBSorT7vaf9MrADSLStU=
        Missing: +MsXx4VfhXEWn1v9uAtD3k1PfC9KjPc4JGws76WRVzA=
        Missing: +WIyQt4V09k6oUmFRXr0Xq3+TDPD3O1nr0odKRoDDTw=
        Missing: +X/40HiVyt2m/3aU2aBxWNYwqoyWOw9oTHVfN7AEwqc=
        Missing: +aqwOtKB5rTSoMF7u+QwWPyElJ9XRWTyNQ5NcsDXsA4=
        Missing: +f7TyW39wxJR401Y7yC+5XrFl1VmCEBqy90rlzgARQI=
        Missing: +jo2VXHo/zBgsZyM0togiiL0ssNmghH4pyyQAxCma1Y=
        Missing: +wGGcxdCIVR9UI+KLjeXARkxQ9ojfbuVVsGGrUuccTM=
        ... and 537 more

duplicati-iac1e95df3d8943cc89d0e26a74f00948.dindex.zip.aes: 12 errors
        Missing: 5Ilsi3DQmlXg/jELiO4P9LRCOU30vsH/1GnKCJ2Dm64=
        Missing: 9eQZqGEELa4QjkZadAhJRm30p0DC8Q4iJtcrATcbe2w=
        Missing: JYmjQkseSKgafs4zHlnXyEehlXsrGyRe8owt+JliqNI=
        Missing: QrkueEEerBcFv6nvr1L330sA6mhhIgTHRvlr7P9VXHo=
        Missing: SH1t/7K3/ooZdNzRTC6YmgaqaDqu1V2HV9sQ2oTkKpk=
        Missing: UjDOQZT8km6/YeM5Vzi/zGHV/9ndz0B6ng0NgJnU1HM=
        Missing: amHAOQJT7MUw+hQoBfRo9z1MQLvGLS/tDwBfsvl883k=
        Missing: gVGdQ7aZirdvrVuhYPSBgRz6fUxL839UR659k2JHJeY=
        Missing: mYdCZMsRQYtU5GH0NNl5k6rrBdIT2hJt8eNh5nlHyy8=
        Missing: x6PpPAC/zJW+0uffX+TuVPSxzrfZswHATzTYqbwiyoo=
        ... and 2 more

Followed later by:

duplicati-bc0bf5280d11f4d579a04bee5b4eb83b1.dblock.zip.aes: 3 errors
        Extra: LwWF46cDsWLehMeNFDdazvTf3ywk4khXB0k9dmDj+Ts=
        Extra: SMBZGirE4fUq7A6dJ0d7kkyAKpTXGHburoOca/TVc9o=
        Extra: z2AnluoLmEAFhgXVOUTO8IsYU6ztfgv9jGdaYaRrOzo=

duplicati-b4b09349f34e34766931ca770d5c16c94.dblock.zip.aes: 1 errors
        Extra: +0nAIzGYf+Dk4B5xdkLE+YwhcJKZTXeQ/edh+51aQ24=

duplicati-b9c9cf6692369429f918d10556d0e997e.dblock.zip.aes: 8 errors
        Extra: EqfhN/yNXsa/W4mOAQ+N5Bp4x+l584JfkPtyLWXw4/g=
        Extra: Fcg2KCk4DSM72/aCzMEkmOHpK5uohqU5ujrck0RQgVM=
        Extra: JUI6HCpKIpOBexLmgZl1J/7b/lHUbdtY2DS0D2ZPd6U=
        Extra: a5ZUAxHH9tNX6/DVmV7bVWd0J5gwRL95PljMVC8owiA=
        Extra: pDs5B8ETZhKUNVYdVngn6TM0xCBUThyTSZqouN5ha9E=
        Extra: rDJKW4cMaioxhi7IcwIh89cY2Nv5Z40x3YreWbzLYFY=
        Extra: rq+tNKWR/xaQ/NwPmy9pgUua5rqiXuDLWo3mh2Qrixk=
        Extra: yaYb73idKyKzOZBiW994Th5zFILyiJVbLeepLOlBosQ=

Both are just small samples of the messages.

How do I fix this?

ts678 · June 5, 2025, 11:52am

Do you know if troubled backup was doing this sort of test output before?
You would have to use full-remote-verification to look at internal data.

Any given repair method presumably has limits, but shouldn’t worsen things.

Some of this reminds me of the harmless “Extra” noise that’s been observed.
The specific block complaints are paired between the dblock and dindex files.
You can see if you see that pattern, however that doesn’t explain a “Missing”.

Beyond any history you give, this probably needs some developer comment.

Taomyn · June 5, 2025, 12:54pm

This is the first time I have tried test so I can’t say about the past, but the errors during backups for these files only started with .119

This is the command I am running to try and fix things:

'C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe' test file://D:\LISA\DUPLICATI\LOCAL\ all --asynchronous-upload-folder=T:\Duplicati\ --tempdir=T:\Duplicati\ --dbpath=C:\ProgramData\Duplicati\ICXFGPFETH.sqlite --dblock-size=100MB --passphrase=<redacted> --full-remote-verification=true

Each time, 3 runs so far, it states the following so nothing is being done to the backup:

Found 59 faulty index files, use the option --replace-faulty-index-files to repair them
Verified 2471 remote files with 59 problem(s)

I did also try purge-broken-files but it never seems to complete so I dare not risky removing the --dry-run:

'C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe' purge-broken-files file://D:\LISA\DUPLICATI\LOCAL\ --asynchronous-upload-folder=T:\Duplicati\ --tempdir=T:\Duplicati\ --dbpath=C:\ProgramData\Duplicati\ICXFGPFETH.sqlite --dblock-size=100MB --passphrase=<redacted> --dry-run

ts678 · June 5, 2025, 1:15pm

I was going by what you said previously, which had me wondering about new damage:

but the latest post didn’t show that, so I’m confused about the difference in descriptions.

Maybe usage changed. If there is a chance you used --replace-faulty-index-files, looking at the file timestamp on any dindex files now reporting missing blocks might help determine whether the replacement result has problems, or if old dindex are affected too.

Taomyn · June 5, 2025, 1:38pm

I checked a few dindex and dblock files that the last run reported as having errors. All the dblock files have quite old timestamps, going back months/years, whereas the dindex files all had new timestamps from about 6hrs ago, probably from an earlier run.

I suppose it’s doing exactly as instructed and only repairing or trying to repair index files. I tried to guess a few parameters to do all files without any luck. The command-line help isn’t telling me much either.

Adding: this is what a repair does:

PS C:\Users\administrator.STAR-ONE> & 'C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe' repair file://D:\LISA\DUPLICATI\LOCAL\ --asynchronous-upload-folder=T:\Duplicati\ --tempdir=T:\Duplicati\ --dbpath=C:\ProgramData\Duplicati\ICXFGPFETH.sqlite --dblock-size=100MB --passphrase=<redacted>
  Listing remote folder ...
The empty index file duplicati-i836147d0459948d4a5d7663528f26aab.dindex.zip.aes is larger than expected (54541 bytes), choosing not to delete it

ts678 · June 5, 2025, 2:59pm

You can look in your job log files for a log from Test which will have its start and end time. That, plus the timestamp on the dindex, should confirm that’s what made it, although usual situations such as backup or even compact would be uploading dindex to match its dblock.

Your dindex-only uploads sound like possibly the result of the fix code. After confirming the situation, question for developer is whether it should be possible to get the result you got…

As an aside, dindex files can be deleted and then replaced by repair, but please hold off, as possibly you found a problem in the new code, or maybe some limitation on its capabilities.

Taomyn · June 6, 2025, 6:17am

No problem to wait for the time being - backups continue to run, sometimes they error but they all throw warnings.