Release: 2.1.0.102 (Canary) 2024-12-12

kenkendk · December 12, 2024, 2:34pm

2.1.0.102_canary_2024-12-12

This release is a canary release intended to be used for testing.

Note that this build is using .NET8, unlike the 2.0.8.1 build

Note that this build has a redesigned restore flow
Should issues arise, the previous restore engine can be enabled with --restore-legacy=true.

Detailed list of changes:

Upgraded FluentFTP to fix a crash on socket timeout
Removed named channels to fix a rare exception
Rewrote the restore process to be multithreaded and target optimal disk I/O write performance
Added missing documentation to the iDrive e2 backend
Added activity timeout on backend transfers
Added support for generating a “forever token”

Taomyn · December 13, 2024, 9:04am

I have upgraded from .101 on all my Win11,Win2025,Debian,Fedora, RPI-OS (Arm7/Arm64) and so far no installation issues and backups seem to be fine.

I did see with the previous Windows canary, all versions, an issue with the tray icon process causing an exception to pop up on-screen when restarting or shutting down the machine. I will post a screen shot if this canary does the same thing.

Taomyn · December 13, 2024, 9:54am

It’s happening, and all I did was sign out this time

kenkendk · December 13, 2024, 1:16pm

I tried this a few times and I could not replicate. Is your setup with a WindowsService and a standalone TrayIcon?

Taomyn · December 13, 2024, 1:34pm

Yes, all are Duplicati as a service using System account, and the tray icon with logged in user account.

ollibraun · December 13, 2024, 3:33pm

I have noticed some strange behavior compared to the previous version (not in all installations, everything here is always a Windows service):

When rebuilding the database, you can see it nicely – I didn’t cancel anything:

ollibraun · December 14, 2024, 6:52am

Some jobs work as before. I think the database rebuild no longer works (it wasn’t a problem in the previous version):

14. Dez. 2024 07:49: Operation Get with file duplicati-b8f3bd1d27b2f4bed8b609b283f1e9cee.dblock.zip attempt 1 of 5 failed with message: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Duplicati.StreamUtil.TimeoutObservingStream.WriteImplAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at Duplicati.Library.Utility.ThrottledStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.<>c.<BeginWriteInternal>b__52_0(Object <p0>)
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Duplicati.Library.Utility.Utility.CopyStreamAsync(Stream source, Stream target, Boolean tryRewindSource, CancellationToken cancelToken, Byte[] buf)
   at Duplicati.Library.Utility.Utility.CopyStreamAsync(Stream source, Stream target, CancellationToken cancelToken)
   at Duplicati.Library.Backend.MicrosoftGraphBackend.GetAsync(String remotename, Stream stream, CancellationToken cancelToken)
   at Duplicati.Library.Main.BackendManager.DoGetFile(FileEntryItem item, IEncryption useDecrypter, CancellationToken cancellationToken)
   at Duplicati.Library.Main.BackendManager.DoGetAsync(FileEntryItem item, CancellationToken cancellationToken)
   at Duplicati.Library.Utility.Utility.Await(Task task)
   at Duplicati.Library.Main.BackendManager.ThreadRun()

ollibraun · December 14, 2024, 6:54am

I was just wondering if cancellation requests that were registered at some point in the past are suddenly being processed. Past means - in an older version, and despite server restarts in the meantime.

ollibraun · December 14, 2024, 7:29am

At the end of a backup job, during the check:

Folgore101 · December 14, 2024, 10:54am

It tells me there is a version 2.1.0.103 but it can’t find it.

Hannes3 · December 14, 2024, 11:11am

The same here:
Not Found

There is something strange in the name of the subdirectory (çanary)

ts678 · December 14, 2024, 12:24pm

I’m still a Canary behind this one, but it looks like the CLI updater broke.
Also confirming a similar issue with the mystery release announcement.

C:\Duplicati\duplicati-2.0.9.106_canary_2024-09-03-win-x64-gui>Duplicati.CommandLine.AutoUpdater check
New version is available: Duplicati v2.1.0.103 - Canary

C:\Duplicati\duplicati-2.0.9.106_canary_2024-09-03-win-x64-gui>cd ..\*2.0.9.107_*

C:\Duplicati\duplicati-2.0.9.107_canary_2024-09-11-win-x64-gui>Duplicati.CommandLine.AutoUpdater check
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
No updates found

C:\Duplicati\duplicati-2.0.9.107_canary_2024-09-11-win-x64-gui>

EDIT:

C:\Duplicati\duplicati-2.1.0.102_canary_2024-12-12-win-x64-gui>Duplicati.CommandLine.AutoUpdater check
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
No updates found

C:\Duplicati\duplicati-2.1.0.102_canary_2024-12-12-win-x64-gui>cd ..\*2.1.0.2_*

C:\Duplicati\duplicati-2.1.0.2_beta_2024-11-29-win-x64-gui>Duplicati.CommandLine.AutoUpdater check
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
Error detected: System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Utility.HttpClientExtensions.DownloadFile(HttpClient client, HttpRequestMessage request, String filename, Action`1 progressReportingAction, CancellationToken cancellationToken)
   at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate(ReleaseType channel)
No updates found

C:\Duplicati\duplicati-2.1.0.2_beta_2024-11-29-win-x64-gui>

https://updates.duplicati.com/canary/latest-v2.manifest does indeed talk about 2.1.0.103.

Hannes3 · December 14, 2024, 2:15pm

I have the same problem. It happens after a period of “sleep”.

(no windows service)

ollibraun · December 15, 2024, 2:20pm

So, here’s what it looks like for me (Windows service, destination almost always SharePoint, upgrade from 2.1.0.101 to 2.1.0.102):

Some jobs work. Others no longer work, and you get a message saying that a task has been canceled, whatever that means – I didn’t click Cancel):

ts678 · December 15, 2024, 2:29pm

Is there a relationship between that and this problem? Unfortunately, I don’t have SharePoint.

EDIT:

shows it failing on some MS destination (now revealed). Does it fail similarly on other storage?

ollibraun · December 15, 2024, 2:59pm

Approximate numbers: I have about 20 targets, 75 percent of which are SharePoint or OneDrive.

The non-Microsoft targets work, but most of the Microsoft targets work too.

Only five Microsoft targets don’t work.

Unfortunately, this is not enough to make a statistical statement as to whether it only affects MS targets.

What does “TaskCanceledException: A task was canceled.” mean?

ts678 · December 15, 2024, 3:39pm

It’s an internal thing. Single process has many tasks inside it. Microsoft:

Task cancellation (but I’m guessing a bit because I don’t write the code)
How to: Cancel a Task and Its Children links to TaskCanceledException

Still looking for anything they might share. You can certainly try more tests with them.
Easy is Verify files button, which (in addition to DB tests) downloads a file sample.

Harder would be GUI Export As Command-line then run BackendTool get some files.
BackendTester is automatic, but you’d have to modify the URL to use an empty folder.

Ultimately this is going to need dev help, but we can poke around to get the first clues.

ollibraun · December 16, 2024, 7:25am

In between:

I think it only happens when compacting SharePoint backups.
On a device with three non-working jobs, I went back to.101. This works for backing up, compacting and rebuilding the database.

Here is the error that apparently occurs during compacting:

kenkendk · December 16, 2024, 10:07pm

And @Taomyn , I have created an issue for the crash message.

ollibraun:

14. Dez. 2024 07:49: Operation Get with file duplicati-b8f3bd1d27b2f4bed8b609b283f1e9cee.dblock.zip attempt 1 of 5 failed with message: A task was canceled.
System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Duplicati.StreamUtil.TimeoutObservingStream.WriteImplAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)

This looks like a problem with the new timeout detection feature. Looking at the code, it might be caused by the retry-backoff feature in SharePoint. If there is a retry wait period, nothing will happen in this period, so the timeout will kick in.

Can you try to set the option --read-write-timeout=1h to allow it to wait 1 hour (as opposed to 30s) ? I would imagine this would fix the issue. I have created an issue to track the premature timeout.

In short it means: a task was started, but then cancelled (read: stopped) before it could complete.

ollibraun · December 17, 2024, 5:27am

Unfortunately, the read-write-timeout option does not change this behavior. It may well be that the abort message always comes after 30 seconds, even if the option is set to one hour.