Release: 2.1.0.116 (Canary) 2025-04-17

kenkendk · April 17, 2025, 5:57pm

2.1.0.116_canary_2025-04-17

This release is a canary release intended to be used for testing.

This version is mostly the same as 2.1.0.115, so please see release notes from that.

Major changes in this version

This is primarily a bug-fix release with only minor changes.

Potentially breaking changes (from 2.1.0.109)

Location of data folder has changed in some cases, refer to previous release for details
Permissions are now applied to the data folder, refer to previous release for details
New user interface, ngclient is the default; there is a button to switch between the two

Detailed list of changes:

Upgrade Minio library from 3.13 to 6.0.4
Fixed delete command handling retention as well as specific version
Fixed an issue with queries that need more than 128 parameters
Updated memory caching and logging interface packages
Fixed an issue with schedules missing in the UI

Ngclient changes:

Fixed a case where dialogs would be missing the confirmation button
Improved the create-folder flow
Improved default handling of retention settings

Taomyn · April 18, 2025, 6:00am

All my machines upgraded without incident.

So far the backups that have run have been clean, including one of the ones that was reporting the SQL error

webipsum · April 20, 2025, 2:26am

Please, why does this message appear?

This happens when I’m going to finish adding a backup.

webipsum · April 20, 2025, 2:40am

The default is like this.

How do I maintain this configuration?

Folgore101 · April 22, 2025, 6:00pm

I did a fresh installation of Duplicates, removing the old version 2.0.9.105 and installed 2.1.0.116.
The backup on HD gives me no problem but the one on OneDrive gives me this error:

2025-04-22 19:46:16 +02 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error InvalidOperationException: The stream was already consumed. It cannot be read again.
2025-04-22 19:46:16 +02 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Backup has failed with error: The stream was already consumed. It cannot be read again. InvalidOperationException: The stream was already consumed. It cannot be read again.

Do you have any suggestions?

kenkendk · April 25, 2025, 1:32pm

There is a small issue with the display of the error, so it shows the request that failed, but not the error message. It will be fixed soon, but for now, you can try to use the “revert to ngax” button and it will show the error message.

You can also use your browser’s DeveloperTools and see the response and error message if you prefer.

It will be fixed in the next canary.

This sounds like OneDrive is now broken. I have created an issue for this.

kenkendk · April 29, 2025, 2:30pm

I tried to reproduce this but have not been able to. I used the backend tester, and made a simple backup with 150mb data, all defaults, and it works as expected.

@Folgore101, if you still see the issue, can you see any more details, like a stack-trace?

Folgore101 · April 29, 2025, 4:50pm

Ok, thank you. I try to do some tests and i update you.

I set up a new backup and changed the name of the folder on OneDrive but i keep getting errors.

2025-04-29 18:51:52 +02 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
InvalidOperationException: The stream was already consumed. It cannot be read again.

2025-04-29 18:51:52 +02 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: L'operazione Backup non è riuscita con l'errore: The stream was already consumed. It cannot be read again.
InvalidOperationException: The stream was already consumed. It cannot be read again.

            {
  "DeletedFiles": 0,
  "DeletedFolders": 0,
  "ModifiedFiles": 0,
  "ExaminedFiles": 17348,
  "OpenedFiles": 17335,
  "AddedFiles": 17335,
  "SizeOfModifiedFiles": 0,
  "SizeOfAddedFiles": 1850669249,
  "SizeOfExaminedFiles": 2042799617,
  "SizeOfOpenedFiles": 1850669249,
  "NotProcessedFiles": 0,
  "AddedFolders": 0,
  "TooLargeFiles": 0,
  "FilesWithError": 0,
  "TimestampChangedFiles": 0,
  "ModifiedFolders": 0,
  "ModifiedSymlinks": 0,
  "AddedSymlinks": 0,
  "DeletedSymlinks": 0,
  "PartialBackup": false,
  "Dryrun": false,
  "MainOperation": "Backup",
  "CompactResults": null,
  "VacuumResults": null,
  "DeleteResults": null,
  "RepairResults": null,
  "TestResults": null,
  "ParsedResult": "Fatal",
  "Interrupted": false,
  "Version": "2.1.0.117 (2.1.0.117_canary_2025-04-25)",
  "EndTime": "2025-04-29T16:51:52.4533221Z",
  "BeginTime": "2025-04-29T16:47:13.2333756Z",
  "Duration": "00:04:39.2199465",
  "MessagesActualLength": 77,
  "WarningsActualLength": 9,
  "ErrorsActualLength": 2,
  "Messages": [
    "2025-04-29 18:47:13 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: L'operazione Backup è iniziata",
    "2025-04-29 18:47:13 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  ()",
    "2025-04-29 18:47:14 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  ()",
    "2025-04-29 18:47:14 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: QuotaInfo - Started:  ()",
    "2025-04-29 18:47:49 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b52566bc7eaa142c885bdd9688331a90b.dblock.zip.aes (49,21 MiB)",
    "2025-04-29 18:47:50 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b474ebe60cb7a4916a7afe9c412121bcc.dblock.zip.aes (49,27 MiB)",
    "2025-04-29 18:47:53 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-ba216fc3a395e4f9f893db86c97bf4ada.dblock.zip.aes (49,33 MiB)",
    "2025-04-29 18:47:53 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b744545cfb6e54b8a9e0fa79db606f34b.dblock.zip.aes (49,35 MiB)",
    "2025-04-29 18:48:24 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Retrying: duplicati-ba216fc3a395e4f9f893db86c97bf4ada.dblock.zip.aes ()",
    "2025-04-29 18:48:24 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Retrying: duplicati-b744545cfb6e54b8a9e0fa79db606f34b.dblock.zip.aes ()",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-ba216fc3a395e4f9f893db86c97bf4ada.dblock.zip.aes (49,33 MiB)",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-b82b7dcd2ec0840e797db860f2144f941.dblock.zip.aes (49,33 MiB)",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.Backend.PutOperation-RenameRemoteTargetFile]: Renaming \"duplicati-ba216fc3a395e4f9f893db86c97bf4ada.dblock.zip.aes\" to \"duplicati-b82b7dcd2ec0840e797db860f2144f941.dblock.zip.aes\"",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b82b7dcd2ec0840e797db860f2144f941.dblock.zip.aes (49,33 MiB)",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-b744545cfb6e54b8a9e0fa79db606f34b.dblock.zip.aes (49,35 MiB)",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-b22af979edb2d44df8481e52e02e52bc9.dblock.zip.aes (49,35 MiB)",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.Backend.PutOperation-RenameRemoteTargetFile]: Renaming \"duplicati-b744545cfb6e54b8a9e0fa79db606f34b.dblock.zip.aes\" to \"duplicati-b22af979edb2d44df8481e52e02e52bc9.dblock.zip.aes\"",
    "2025-04-29 18:48:34 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b22af979edb2d44df8481e52e02e52bc9.dblock.zip.aes (49,35 MiB)",
    "2025-04-29 18:49:05 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Retrying: duplicati-b82b7dcd2ec0840e797db860f2144f941.dblock.zip.aes ()",
    "2025-04-29 18:49:05 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Retrying: duplicati-b22af979edb2d44df8481e52e02e52bc9.dblock.zip.aes ()"
  ],
  "Warnings": [
    "2025-04-29 18:51:50 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerHandlerFailure]: Error in handler: The stream was already consumed. It cannot be read again.\r\nInvalidOperationException: The stream was already consumed. It cannot be read again.",
    "2025-04-29 18:51:50 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeWhileActive]: Terminating 3 active uploads",
    "2025-04-29 18:51:50 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeWhileActive]: Terminating 3 active uploads",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: upload was active during termination, but had state: WaitingForActivation",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: Terminating, but 3 active upload(s) are still active",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: upload was active during termination, but had state: WaitingForActivation",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: Terminating, but 3 active upload(s) are still active",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: upload was active during termination, but had state: WaitingForActivation",
    "2025-04-29 18:51:51 +02 - [Warning-Duplicati.Library.Main.Backend.Handler-BackendManagerDisposeError]: Terminating, but 3 active upload(s) are still active"
  ],
  "Errors": [
    "2025-04-29 18:51:52 +02 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error\r\nInvalidOperationException: The stream was already consumed. It cannot be read again.",
    "2025-04-29 18:51:52 +02 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: L'operazione Backup non è riuscita con l'errore: The stream was already consumed. It cannot be read again.\r\nInvalidOperationException: The stream was already consumed. It cannot be read again."
  ],
  "BackendStatistics": {
    "RemoteCalls": 19,
    "BytesUploaded": 0,
    "BytesDownloaded": 0,
    "FilesUploaded": 0,
    "FilesDownloaded": 0,
    "FilesDeleted": 0,
    "FoldersCreated": 0,
    "RetryAttempts": 16,
    "UnknownFileSize": 0,
    "UnknownFileCount": 0,
    "KnownFileCount": 0,
    "KnownFileSize": 0,
    "KnownFilesets": 0,
    "LastBackupDate": "0001-01-01T00:00:00",
    "BackupListCount": 0,
    "TotalQuotaSpace": 1104880336896,
    "FreeQuotaSpace": 1104772962713,
    "AssignedQuotaSpace": -1,
    "ReportedQuotaError": false,
    "ReportedQuotaWarning": false,
    "MainOperation": "Backup",
    "ParsedResult": "Success",
    "Interrupted": false,
    "Version": "2.1.0.117 (2.1.0.117_canary_2025-04-25)",
    "EndTime": "0001-01-01T00:00:00",
    "BeginTime": "2025-04-29T16:47:13.2333769Z",
    "Duration": "00:00:00",
    "MessagesActualLength": 0,
    "WarningsActualLength": 0,
    "ErrorsActualLength": 0,
    "Messages": null,
    "Warnings": null,
    "Errors": null
  }
}

Also on version 2.1.0.117 i find problems, what are “stack-trace” and where can i find them?

Folgore101 · May 1, 2025, 12:14pm

Hi @kenkendk i have done some tests and it seems that the problem depends on the size of the source files.
With 150mb actually the backup finishes correctly, the problem seems to occur when the data is over 600mb.

Even if from that error still charged the parts on OneDrive, and the third attempt ends the backup, even if I don’t know how reliable it is.

I’ll also point out these differences between new UI and old UI:

The backup made on 30/04/2025 is identified as:

New UI - “Last successful backup: a month ago”.
Old UI - Last successful backup: Yesterday at 8:31 PM;
It seems more correct to indicate in the old UI.
Even if I do one today he leaves me “a month ago”.

The information popup should be larger, like the one in the image comes out of the browser window.

image579×594 48.5 KB
In Backup destination:

New UI - “The authorization code:” is not clickable;
Old UI - “AuthID:” is clickable and opens the page for authorization.
Better the old UI.

ts678 · May 1, 2025, 1:11pm

This is being tracked in the ngclient issue below:

Last successful backup shows “a month ago” #165

kenkendk · May 3, 2025, 5:53am

The upload size does not depend on the source size? What setting do you have for --dblock-size (aka Volume Size in the UI)?

I have not tested the limits of OneDrive, but perhaps it does not like 500mb+ files?

I have registered an issue for that.

I have added an issue for that.

Folgore101 · May 3, 2025, 1:08pm

I have the default limit of 50 MByte, what i was using even before the upgrade.

I did the test with the folder LibreOfficePortable (603MB) which is part of Portable App

The first attempt created the last two files with the usual error;
Second attempt created the other files and was successful.

Thank you for the other reports.

ts678 · May 6, 2025, 5:57pm

Works fine here on new backup of a 671 MB folder, although to get your mix of alternating dblock and dindex, I had to use Advanced option --asynchronous-concurrent-upload-limit=1. Also works with the default 4, except then I get multiple dblock going up at once.

Do you use any other advanced options that might be relevant?

Sami_Lehtinen · May 7, 2025, 5:59am

Don’t know which version exactly changed what. But I know I complained about transactions (SQLite3) and FTPS session-resumption errors + FTPS opening one unnecessary session, which wasn’t reused properly.

All of those issues are gone with this version. (Yeah, I’m aware there’s even newer one) but this is the one where I had time to check the logs now properly on both sides. FTPS server and Duplicati client side.

Great!

Folgore101 · May 7, 2025, 11:21am

Hi, i created “Test-BK” with just that folder and the default settings, the only changes are in the advanced options that i put --snapshot-policy=Auto
and

ts678 · May 7, 2025, 11:45am

Also without any option adds in Settings default options?

Is this fresh backup of folder that isn’t changing during it?

Regardless, I’m not sure what can be done without repro.

Maybe devs have some further suggestions on the error?

Folgore101 · May 8, 2025, 5:19pm

Yes.

I tried without the option --snapshot-policy=Auto
On the first try as in the previous post;
On the second try now it gives me this error: Detected 1 volume with missing filesets: VolumeId = 1, Name = duplicati-20250508T112228Z.dlist.zip.aes, State = Temporary and it doesn’t go any further.

I don’t think it depends on the removal on the option --snapshot-policy=Auto, anyway over the weekend if i can uninstall everything and try to install at beta version and see if it works.

jberry02 · May 9, 2025, 9:30pm

Is there any update on the issue with the b2 backend (and maybe others?) where upload errors are not getting detected? I am still seeing the “missing file(s)” error every few days with .116.

kenkendk · May 12, 2025, 1:11pm

I tested as well as @ts678.
I downloaded a few files from PortableApps (5 files, +700mb in total), and used that as the source. Destination is OneDrive (not the Business one), encrypted upload, disabled schedule, no other settings changed.

It worked as expected

Some secret sauce is missing I think. Could it be some network stuff?

That sounds strange. A file with State = Temporary is safe to remove, so it should just be removed. Do you have a full error message (preferably with a stack trace)?

You can just remove it from the database if you want to recover, but I would like to know the sequence of events that causes it.

There was a bug with 2.1.0.4 stable, which prompted the release of a 2.1.0.5 release.
I am not aware of any issues with “missing files” for 2.1.0.116 or later.

Can you elaborate on what happens?
Could you look in the database for the tables Remotevolume and RemoteOperation to see if you can track the file that is suddenly missing?

You are also welcome to create a bugreport as soon as you see the error, then I can try to track the sequence.

jberry02 · May 12, 2025, 3:20pm

This is the bug that showed up several Canary releases ago. The recommended work-around was to set --read-write-timeout to 10 minutes. I have it set to 5 minutes, so if there is a significant difference going from 5 to 10 it could be why the work-around isn’t working for me.

I will see if I can find the prior topics, but a quick search so far shows I was experiencing this with .110.