Release: 2.1.0.110 (Canary) 2025-02-28

2.1.0.110_canary_2025-02-28

This release is a canary release intended to be used for testing.

Major changes in this version

  • Restore improvements
    Building further on the new restore engine, this version has significant performance speedups during restore.

  • New signing keys
    The builds are now signed by “Duplicati Inc” for MacOS and Windows.
    The Windows signatures are now done with an extended validation (EV) certificate and should not trigger Windows Defender warnings.

  • New enumeration logic
    The enumeration of files and folders has been rewritten to better support some edge cases.

  • Improved pagination support
    All backends are updated to support the new list methods that supports pagination and cancellation.
    The test method will now only load a single page of results, instead of listing all items.

  • Support for backend-as-source
    This version has support for backups of both remote and local data.
    In this version, S3, IDrive, SSH and CIFS sources are supported.
    The UI does not yet support this, but it is possible to read data from a remote location by “mounting” it into the local filesystem.
    For the commandline (and manual text entry in the UI) enter sources such as:

// Linux/MacOS
@/mnt/s3-data|s3://example?auth-username=...

// Windows
@X:\server1|cifs://server/share?auth-username=...

Potentially breaking changes (from 2.1.0.109)

  • Location of data folder has changed in some cases, refer to previous release for details
  • Permissions are now applied to the data folder, refer to previous release for details
  • New user interface, ngclient is the default; there is a button to switch between the two

Detailed list of changes:

  • Improved script examples
  • Fixed double-click TrayIcon on Windows, thanks @Alex-Heard
  • Fixed an issue with CORS headers
  • Parallel downloads for restore
  • Report interface and port in Webserver startup message
  • Fixed missing example script on Windows
  • Fixed a case where a leading slash could be stripped from the path or a URL
  • Added additional Infomaniak endpoints, thanks @ochaze
  • Backend listing is now async for better pagination and cancellation support
  • Updated signing support for the release builds
  • Added a link from ngax to ngclient (the other way already exists)
  • Added alternate source providers and new enumeration process
  • Performance optimizations for restore process
3 Likes

First time here, so my apologies if this is not the right place to ask for support on this particular version. Just did a version jump from from 2.1.0.109 to 2.1.0.110 via a Docker install and the new default UI (ngclient) display a blank or empty screen.

I am able to change the URL to point to /ngax and the UI loads fine. Am I missing something? I couldn’t find anything in the documents or breaking changes from .109 and .110 on what I need to do to make the ngclient UI work.

I have updated all my machines, Win2025/Fedora-41/RPi-4 without any major issues - I’ve not done any backups yet.

Only thing to report so far is with the new GUI, regarding the language again, although I can now see the dropdown correctly, I cannot use my mouse to choose anything, I have to use the cursor keys then press enter. I still find that “en-GB” is having no effect:

It’s the same for date formats. In case it helps, I use Brave browser but all my local connections are excluded from Brave Shields so there should be no interference from it.

That sounds odd. Any special setup on your side?
Do you see any errors in the browser Developer Tools ?

There is nothing special needed (at least that is the intention).
If you simply cannot get ngclient to work, you can go to /theme.html and set ngax as the default theme, and it will load this one for the next 90 days.

That should be fixed in master. Next release should have searchable dropdowns as well.

Ok, thanks for reporting, I thought it was fixed. I have created a new issue.

1 Like

I’m chalking it up to something with Google Chrome and some cache issue. .209_canary worked just fine and I didn’t look through the Developer Tools. Good idea and I should of, next time I will. Now that there is a stable release, I moved my environment to this instead this week so I don’t have .210 up and running right now.

I am seeing the same thing on a Windows 10 system using Firefox from a Linux system as the browser. I am not seeing the issue on a Linux system (again using Firefox on a Linux system as the browser). In all cases Duplicati is running as a system service on port 8200.

Further testing shows the problem also exists when using chromium and seamonkey (again, browser on Linux system, Duplicati on Windows 10 system).

In all the above cases changing the URL to /ngax/ brings up the prior interface.

Adding in my environment, I am using Chrome on Windows 11 to access Duplicati’s WebUI. Server is running from a Docker container on a Debian 12 system. I did the same thing by just replacing /ngclient with /ngax and the WebUI would work fine.

The .NET 8 releases (even before ngclient) always gave me kind of a flaky GUI, but my daylight saving time test last night really set it off a lot, so I’m looking into it some more.

There’s an image there of stuck status bar and some obsolete times in the screen body. Second open tab didn’t get a stuck status bar, but did get the stuck body. It’s not polling.

First tab is polling progressstate every 2 seconds, except for 401 Unauthorized which it clears with successful refresh. Server is returning information such as the following to it:

{
    "BackupID": "1",
    "TaskID": 109,
    "BackendAction": "Get",
    "BackendPath": "duplicati-ba946be2b3a544e908afcc6e6a95469bc.dblock.zip",
    "BackendFileSize": 770,
    "BackendFileProgress": 0,
    "BackendSpeed": -1,
    "BackendIsBlocking": false,
    "CurrentFilename": null,
    "CurrentFilesize": 0,
    "CurrentFileoffset": 0,
    "CurrentFilecomplete": false,
    "Phase": "Backup_Complete",
    "OverallProgress": 1,
    "ProcessedFileCount": 1,
    "ProcessedFileSize": 118,
    "TotalFileCount": 1,
    "TotalFileSize": 118,
    "StillCounting": false
}

After having captured the image, I went exploring to see (short of refresh) what works.
I haven’t hit anything yet that is refusing a look, but the status is amusingly conflicting:

About → System info in the first stuck tab has:

lastEventId : 233
lastDataUpdateId : 59
lastNotificationUpdateId : 0
estimatedPauseEnd : "0001-01-01T04:56:02.000Z"
activeTask : {"Item1":37,"Item2":"1"}
programState : Running
lastErrorMessage :
connectionState : connected
connectionAttemptTimer : 0
failedConnectionAttempts : 0
failedAuthAttempts : 0
lastPgEvent : {"BackupID":"1","TaskID":119,"BackendAction":"Get","BackendPath":"duplicati-ba946be2b3a544e908afcc6e6a95469bc.dblock.zip","BackendFileSize":770,"BackendFileProgress":0,"BackendSpeed":-1,"BackendIsBlocking":false,"CurrentFilename":null,"CurrentFilesize":0,"CurrentFileoffset":0,"CurrentFilecomplete":false,"Phase":"Backup_Complete","OverallProgress":1,"ProcessedFileCount":1,"ProcessedFileSize":118,"TotalFileCount":1,"TotalFileSize":118,"StillCounting":false}
updaterState : Waiting
updateDownloadLink :
updatedVersion :
updateDownloadProgress : 0
proposedSchedule : [{"Item1":"1","Item2":"2025-03-09T03:40:00Z"},{"Item1":"2","Item2":"2025-03-09T04:30:00Z"},{"Item1":"3","Item2":"2025-03-09T05:15:00Z"}]
schedulerQueueIds : []
pauseTimeRemain : 0

and in the second stuck tab (which doesn’t appear to be polling, per Edge dev tools):

lastEventId : 619
lastDataUpdateId : 129
lastNotificationUpdateId : 0
estimatedPauseEnd : "0001-01-01T04:56:02.000Z"
activeTask :
programState : Running
lastErrorMessage :
connectionState : connected
connectionAttemptTimer : 0
failedConnectionAttempts : 0
failedAuthAttempts : 0
lastPgEvent :
updaterState : Waiting
updateDownloadLink :
updatedVersion :
updateDownloadProgress : 0
proposedSchedule : [{"Item1":"2","Item2":"2025-03-09T11:30:00Z"},{"Item1":"1","Item2":"2025-03-09T11:40:00Z"},{"Item1":"3","Item2":"2025-03-09T13:15:00Z"}]
schedulerQueueIds : []
pauseTimeRemain : 0

Although it’s not talking to server much (after quite awhile), I see it did do something:

proposedSchedule seems to line up with the main screen, but is far from current now.

The console for batch file which launched the TrayIcon is quiet, with no errors shown.

EDIT 1:

The last comment I got on this was here, with a theory about refresh tokens. I poked first stuck tab pretty hard, and shown evidence that it’s not stuck, and able to do new refresh.

I’m not sure where second stuck tab is at, but will leave it alone for awhile. I had a third tab for debugging looks for a little while. It was doing backups requests occasionally to presumably update its main screen. The scheduled DST test jobs are continuing to run.

The stuck first tab unsticks its status bar briefly. A 20-minutes-after-the-hour job just ran and status bar flashed “Waiting for upload to finish”. Screen body has yesterday’s run…

EDIT 2:

Seemingly nobody is available to look, and Windows Update needs a reboot soon.
I will continue poking, and not worry much if some state gets lost, as it will anyway.

Not much has changed though. First stuck tab remains inconsistent in status data:

It actually has 252 versions, the last one this morning. I’ll see if refresh solves this.
It does, so not much left to do with this one. Close it to quiet things for second tab.

Traffic (at least per Edge) is still missing since acknowledgements three days ago.
Refresh fixed that, and things appear back to normal. GUI’s logs of DST seem OK.
What was expected was that at 2 A.M, local time would “Spring forward” to 3. A.M.

There are still no errors on the terminal output. If repro of GUI hang is ever wanted, I’d give it a pretty good chance, since it was easy to fall into (and luckily get out of).

On my Windows 10 system running "Duplicati - 2.1.0.110_canary_2025-02-28
" with backups stored on B2 (native interface), last night’s backup failed.

The error displayed is "Found 2 files that are missing from the remote storage, please run repair
"

Clicking on “Repair” results in the message:
“”"
The remote files are newer (3/12/2025 6:00:00 AM) than the local
database (3/12/2025 2:00:00 AM), this is likely because the
database is outdated. Consider deleting the local database and
run the repair operation again.
“”"

I have been seeing this on an earlier Canary release and was hoping this was fixed in 2.1.0.110. In general I found that the various suggested actions in the prior release didn’t actually resolve the problem. As I recall sometimes I was able to use “purge-broken-files” followed by deleting the failed backup, and other times I had to delete and recreate the database.

Are there any recommended actions or troubleshooting steps I can take to help resolve the bug as well as fix my backups?

Canary releases are testing releases. Problems should be reported to allow fixes.
Please check for existing reports of course. Thanks for this report. Cause may be:

which went into 2.1.0.109 and seems to have some time zone confusion. I can do:

image

in the UTC-4 timezone, and have Repair say:

The remote files are newer (3/12/2025 2:30:00 PM) than the local database (3/12/2025 10:30:00 AM), this is likely because the database is outdated. Consider deleting the local database and run the repair operation again.

Remote file times are 4 hours ahead, likely due to UTC. Local 2:30 isn’t here yet.
I’m not sure what time zone the unit tests run in, e.g. UTC+1 might not see this…

EDIT 1:

I’m not certain about this (need developer help), but the Canary and Beta releases lack some of the urgent fixes that led to 2.1.0.5 Stable. Not sure if that will be fixed.

C:\Duplicati\duplicati-2.1.0.110_canary_2025-02-28-win-x64-gui>Duplicati.CommandLine help read-write-timeout
  --read-write-timeout (Timespan): Set the read/write timeout for the connection
    The read/write timeout is the maximum amount of time to wait for any activity during a transfer. If no activity is detected for this period, the connection is considered broken and the transfer is
    aborted. Set to 0s to disabled
    * default value: 30s

has caused silent upload failures. 2.1.0.5 set that to a larger time (but not disabled):

Possibly a manual config (maybe in Settings, to be a default for all jobs) would help.

As for putting you backup back together, Repair will probably be willing to run when its time confusion ends, but it might not help. You can try to find what variety of files are missing with About → Show log → Live → Warning and click on Verify files. Timeouts (if that’s the issue) are probably unfortunately more likely on dblock files.

Duplicati didn’t stop on dblock Put fail, and Put dlist using it. Later on, some successful B2 Put didn’t actually leave files

is me hitting the timeout problem on 2.1.0.107. I do have some ideas on recovering, and might try them soon (getting tired of downtime). If they work, maybe you can try.

From some additional testing:

  • if I am local to the Windows 10 system and access it as “localhost:8200”, it works (although I think there is stuff missing from the “next backup” message). It also prompted me for the password.
  • if I am local to the Linux system and access it as “localhost:8200”, it works. (The “next backup” box says "- in 11 hours at 2:00 AM " - i.e. it doesn’t say what happens in 11 hours. I’m reasonably sure this was the same issue on the Windows 10 system.) Note the “ngclient” UI never prompted me for the password; it relied on cached/saved data.
  • if I am local to the Linux system and access it as “hostname:8200”, it fails only displaying the blank screen

Maybe the issue is with displaying the new UI when remote? Or maybe only when remote and prompting for the password?

Couple of additional comments on the new GUI

  • Will the live log come back or has it moved elsewhere?

  • Could “today” and “yesterday” be used again for “Last successful backup”, and the “Next schedule run”?


image
It makes it very quick to see at a glance if backups have not run when they should have or about to

Did you fix your backup? If so, how? I asked questions but got no response.
I was thinking that your failure might be less messy than mine was though…
Mine was a double failure. If yours is missing dblock files, purge-broken-files probably is the first thing to try, and maybe delete of that version is a second.

Sorry for the delay… been a bit slammed lately.

Note - ngax UI.
Repair fails - silently, but perhaps that is because I had not yet dismissed the dialog box?
purge-broken-files gives very little output - specifically it does not list any files that it is deleting.
list-broken-files gives:
“”"
Listing remote folder …
Return code: 0
“”"

Next step I know of to get it working is a database “Delete and Recreate” followed by deleting the bad backup. This is in progress.

EDIT 1 - database recreated, new backup run, bad backup deleted. Added --read-write-timeout of 5 minutes.

On my Linux system running 2025-02-28 - 2.1.0.110_canary_2025-02-28…

The new UI did not provide any indication the backup had failed other than the date/time of the last good backup and a small red bell icon[?] which I overlooked at first. I was able to use the 3-dot menu to get to the log, which shows:

warning-circle Warnings 1

`"2025-03-13 02:07:42 -04 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-20250313T060000Z.dlist.zip.aes"`

and

warning Errors 3

`"2025-03-13 02:07:42 -04 - [Error-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteFiles]: Found 1 files that are missing from the remote storage, please run repair"` `"2025-03-13 02:07:42 -04 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error\nRemoteListVerificationException: Found 1 files that are missing from the remote storage, please run repair"` `"2025-03-13 02:07:42 -04 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Backup has failed with error: Found 1 files that are missing from the remote storage, please run repair\nRemoteListVerificationException: Found 1 files that are missing from the remote storage, please run repair"`

Side question - with the new UI how does one get to the “live log”?

The repair operation appears to be good - it uploaded a dlist file:

`  "Messages": [
    "2025-03-13 09:58:42 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started",
    "2025-03-13 09:58:45 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  ()",
    "2025-03-13 09:58:52 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (14.638 KiB)",
    "2025-03-13 09:58:59 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-20250313T060000Z.dlist.zip.aes (28.481 MiB)",
    "2025-03-13 09:59:01 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-20250313T060000Z.dlist.zip.aes (28.481 MiB)"
  ],`

At this point the backup appears to be running - it is counting files.

EDIT 1 - backup completed. I have added --read-write-timeout and set it to 5 minutes. I used the ngax interface to do this because the new interface needed more clarity as to what value should be set; i.e. if I entered “5” is that minutes or seconds?

Hi.

  1. In the settings, there is no Brazilian Portuguese language in the drop-down menu. Is there a plan to include it?
  2. I have 4 backups configured. When I run a backup, the date/time does not appear automatically. I need to refresh your browser to see the date/time of the backup performed.
    Tks.

I edited my earlier posts, but summarizing here for completeness.

2 systems, both now fixed.
Windows system: repair failed, purge-broken-files did nothing, had to do a database “Delete and rebuild”
Linux system: repair worked

On both systems a new backup was run and reported no errors, the bad backup sets were deleted, and --read-write-timeout was set to 5 minutes in the hope of alleviating the problem until a proper fix is available.

2 Likes

Do scheduled jobs override any other running non-job tasks?

I ask because I was running a Repair (database delete and recreate) for my Wasabi S3 storage job, and it was going to take a few days as it has done before.

I left it last night running but when I checked this morning, the Repair had gone and it had run the scheduled daily backup, that then failed (Unexpected difference in fileset version 2). I can’t find any log for the Repair job so I have no idea if it succeeded, failed or was simply killed by the scheduled job.

I’ve restarted the Repair now, but also removed the schedule on the daily back to be sure it doesn’t do the same again.

What’s the oldest log there? If pre-Recreate, maybe Recreate didn’t actually start.

I still find the UI not quite reliable, and ngclient is worse.

You can sometimes also find useful messages in server log at About → Show log.

I had my first view of an ngclient status bar (with moving progress bar as well) on
Restore (delete and repair). Repair said Recreating database or similar, seemingly wrong for Repair. For some reason status bar reverted to just Loading

While I was trying to look at the wrong Repair status, I used Edge developer tools. Interesting it only showed repair I’m not sure why. I used Wireshark and got lots more, including the below which caused me to finally file a Repair issue in GitHub.

GET /api/v1/notifications HTTP/1.1
Host: localhost:8110
Authorization: Bearer REDACTED
User-Agent: Duplicati-TrayIcon-Monitor/2.1.0.110


HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 14 Mar 2025 11:58:09 GMT
Server: Kestrel
Transfer-Encoding: chunked

60a
[{"ID":1,"Type":"Error","Title":"Error while running test 3","Message":"The remote files are newer (3/12/2025 2:20:41 PM) than the local database (3/12/2025 10:20:41 AM), this is likely because the database is outdated. Consider deleting the local database and run the repair operation again.","Exception":"Duplicati.Library.Interface.UserInformationException: The remote files are newer (3/12/2025 2:20:41 PM) than the local database (3/12/2025 10:20:41 AM), this is likely because the database is outdated. Consider deleting the local database and run the repair operation again.\r\n   at Duplicati.Library.Main.Operation.RepairHandler.RunRepairRemote(IBackendManager backendManager, CancellationToken cancellationToken)\r\n   at Duplicati.Library.Utility.Utility.Await(Task task)\r\n   at Duplicati.Library.Main.Operation.RepairHandler.Run(IBackendManager backendManager, IFilter filter)\r\n   at Duplicati.Library.Main.Controller.<>c__DisplayClass26_0.<Repair>b__0(RepairResults result, IBackendManager backendManager)\r\n   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`2 method)\r\n   at Duplicati.Library.Main.Controller.RunAction[T](T result, IFilter& filter, Action`2 method)\r\n   at Duplicati.Library.Main.Controller.Repair(IFilter filter)\r\n   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)","BackupID":"3","Action":"backup:show-log","Timestamp":"2025-03-14T11:58:09Z","LogEntryID":null,"MessageID":"RemoteFilesNewerThanLocalDatabase","MessageLogTag":null}]
0

This proves that the Repair refusal doesn’t go away when local time catches UTC.