Release: 2.3.0.101 (Canary) 2026-05-04

[kenkendk]

2.3.0.101_canary_2026-05-04

2026-05-04 - 2.3.0.101_canary_2026-05-04

This release is a canary release intended to be used for testing.

Changes in this version

Zstd Compression Support

Added a new experimental compression module that supports the zstd compression algorithm, providing faster and more efficient compression for backups.
This new format is currently only for testing and will log a warning when used. Please do report feedback on speed or usability for this feature.

SharpAESCrypt v3

Updated the SharpAESCrypt encryption library to support “AES Crypt Stream Format v3”, which has a number of improvements over the v2 format.
For this release, the default written format remains v2, but we encourage you to set the environment variable DUPLICATI__AES_VERSION=3 to test the new format.
Note: if you set this version to 3, the new remote volumes cannot be read by Duplicati versions older than this version (2.3.0.101).

Post-Backup Script Support

The RunScript module now supports running scripts when a backup has finished, but before checks and compaction.
With this new script option it is possible to pause/resume critical functions for a shorter period.

Remote Synchronization Improvements

Added destination space quota checks to the remote synchronization runner, helping prevent failures due to insufficient destination space.
An option to disable quota checks (quota-disable) has also been added for scenarios where quota information is not available or reliable.
The remote sync code now handles backend instantiation failures more gracefully.

Maintenance and Recompress Fixes

Fixed an issue where the write-path option was not correctly persisted and restored during maintenance and recompress operations, thanks @aureliandevel.

More Defensive Delete Operations

Added additional defensive checks during the delete operation to prevent database constraint violations in edge cases.

Certificate Chain Preservation

The web server again returns full certificate chains when serving HTTPS.

Avalonia Update

Updated the Avalonia UI framework to version 12.0.2, which no longer requires DX12.

Detailed Changes

• Added experimental zstd compression support
• Updated SharpAESCrypt to version 3
• Added post-backup script support to the RunScript module
• Added destination space quota checks to remote synchronization
• Added option to disable quota checks in remote synchronization
• Improved S3 backend to use precomputed hashes for uploads
• Added content length support for non-seekable streams in S3 backend
• Improved error handling in remote sync
• Fixed persisted write-path option for maintenance and recompress operations, thanks @aureliandevel
• Added more defensive checks during delete operations
• Fixed certificate chain preservation in HTTPS server
• Added support for DO_NOT_TRACK environment variable
• Added logging of 500 errors to live log
• Fixed SecretProvider initialization failure handling
• Improved expired remote control link handling
• Fixed full-result option mismatch with alias support, thanks @aureliandevel
• Added remote-volume-size option alias
• Updated Avalonia to version 12.0.2
• Updated all localizations, thanks to all translators

ngclient changes

• Removed flicker on console signup
• Expanded restore view to work better on large screens
• Fixed filters on Commandline page not being fully editable
• Added filename searching to the restore page
• Fixed an issue where a folder would not always expand when loading the tree
• Fixed an issue where invalid requests would be sent to the backend when loading the tree
• Updated to latest ShipUI
• Fixed an issue with referencing crypto API directly

[ItEndsWithTens]

I’m very excited about the new zstd support, but one question: is it expected that after changing compression type with the Recovery Tool’s ‘recompress’ feature, a database rebuild does not change the compression-module value for a given backup in Duplicati-server.sqlite? It doesn’t change for me, and it seems to be the root of a problem.

I’ve been converting my backups to the new compression format over the past couple days (my less mission-critical backups, anyway, since I understand zstd is thus far only experimental) and have run into an issue with the Compact command line option. It doesn’t seem to be caused by Compact itself, nor by the use of the command line in general, but by the fact that the database recreate process puts a ‘compression-module’ key in the Configuration table of a backup’s local database with the wrong associated value. That, in turn, appears to originate in the Option table of %APPDATA%/Duplicati/Duplicati-server.sqlite.

The Compact command doesn’t make for a good repro, because I don’t know how to easily and quickly produce a backup with some amount of wasted space on the destination. But it seems the Purge command, used with --dry-run, demonstrates the issue well enough, so try this:

1. Create a new backup of some small dataset, just a few files is fine, and no encryption is necessary. Make note of one of the filenames, its full path actually.
2. Run the backup once.
3. Delete the backup’s database.
4. Use the Recovery Tool to recompress the Deflate format .zip files Duplicati produces by default to the new ZStandard .tzstd files: “C:\Program Files\Duplicati 2\Duplicati.CommandLine.RecoveryTool.exe” recompress tzstd <remoteurl> <localfolder> --compression-module=tzstd --reuploadReplace remoteurl and localfolder as appropriate on your system.
5. Back in the web GUI, recreate the database deleted in step 3.
6. On to the command line page for the backup in question, choose the ‘purge’ command, and enter the filename from step 1, with full path, in the Commandline Arguments field.
7. Scroll down to the Advanced options, and see there are already signs of a problem: the “Select what module to use for compression” option is still set to ‘zip’ even though there were no zip files in the destination directory when the database was recreated.
8. Still in the advanced options, add the “Do not perform any modifications” option and toggle it to enabled (or just edit as text and add --dry-run after the existing options).
9. Run the purge command, and at the bottom you should see something like “Would upload duplicati-timestampZ.zip (X bytes) and delete file duplicati-timestampZ.tzstd”

All of that despite the recompression step succeeding, and putting .tzstd formatted files on the destination. The backup also runs and restores without issue, for some reason, but attempting certain operations will fail with an error about attempting to change the compression-module from zip to tzstd.

If you follow the above instructions up through step 4, but then manually edit Duplicati-server.sqlite to change the compression-module option’s value from zip to tzstd and restart Duplicati, you can proceed with step 5 and onward and everything should work as expected.

Is this just an oversight from the zstd update, or is there an alternative workflow expected of me when converting compression formats?

I’m otherwise thrilled with this update! I had been converting some of my backups to the zstd+zip format introduced shortly before this one, but the .tzstd version of the idea has proven meaningfully more efficient. One of my more compression-friendly backups, my programming projects directory which is mostly plaintext code files (I filter out most binary build artifacts for this backup config), has dropped from a ~39.9GiB destination size with deflate+zip level 9 to ~35.2 at zstd+tzstd level 15, which I find more than satisfactory. The zstd+zip version was somewhere between the two. Since I pay for cloud storage per gigabyte per month, shaving multiple gigs off of one backup is nice to see. It’s pennies, but they add up quick for those of us on a budget.

Thank you for the work so far! The new format looks rather promising, to say the least.

[kenkendk]

@ItEndsWithTens

Is this just an oversight from the zstd update, or is there an alternative workflow expected of me when converting compression formats?

Since we have not had multiple compression formats for a while, there is probably some code somewhere that is lazily expecting zip compression to be picked. I will try to reproduce the issue.

Edit: Found the issue. It happens if you run recreate without setting the --compression-module option. I have a fix for it here:

Persist used compression module (#6901)

@ItEndsWithTens

The new format looks rather promising, to say the least.

Amazing!

One worry that I have is that we cannot guess the size of the archive in advance as we only start compressing once the volume is filled. In theory, this could create some “jagged” size volumes that are significantly under the target (say volume size is 100MB, but some volumes are 1MB in size). If this happens, it could trip the auto-compact to think that the volumes could be combined, only to end up making the same volumes again and again.

Do you see large variations in size?

[ItEndsWithTens]

@kenkendk

Found the issue. It happens if you run recreate without setting the --compression-module option.

I’m sorry to say I’m still seeing the issue when following the steps I provided, but it looks like it’s related to the new code changes only persisting the compression-module option when it’s either missing or blank.

In your new unit test, you create a Dictionary used to initialize the options, wherein you specify the “no-encryption” key and its value. Debugging that test, I see that by the time we get to VolumeReaderBase.cs, and it’s time to check the compression-module key, it’s nonexistent, and everything works fine.

Building the tray icon executable from the latest code and debugging that, however, then following the repro steps from my previous post (not reusing any of my old existing backups, or previous test configs, but starting fresh), I see that by the same point, the compression-module key is already present, and set to “zip”, so it remains untouched and the problem persists.

If I modify that unit test to also add a “compression-module” key with the value “zip”, right alongside “no-encryption” when initializing that Dictionary, the unit test fails, and the recreated database retains the zip compression setting.

@kenkendk

Do you see large variations in size?

Yes, unfortunately. I ran compact on my aforementioned development backup overnight last night, and this morning found that although the compact reduced the 35.24 GiB total size to 35.09 GiB, it ballooned the file count from 1,656 files before, to 3,534 after. 14 dlist files, 1,760 dblocks, and another 1,760 dindex.

All expected versions are present in the backup, I can browse them just fine, and restore spits out the backed up files successfully. The dblock files at the destination are also all under the 50 MiB size I’ve been using, with the largest being 51,079 KiB, but fewer than 100 of the dblocks are close to that size. The rest are smaller, and it’s a strangely smooth gradient all the way down; I’ve got a few 49.X MiB files, then some 48.X MiB, then 47.X, 46.X, and so on and so on all the way down to 1.X MiB without skipping any. That is, for any number N from 1 through 49, I’ve got a dblock that’s N.X MiB in size. I’m not sure if that’s indicative of anything, but it’s food for thought, at least.

Edit: To be clear, that’s after a Compact. I no longer have the pre- and post-recompress files for the development directory backup I’ve been referencing, but looking at another backup I’m recompressing, I can offer details of just the conversion process.

For this other backup, my previous deflate+zip version had 893 dblock files, all but 2 of which were between 42.0 and 50.0 MiB. The 2 outliers were 33.3 and 23.7 MiB.

After recompressing to zstd+tzstd, I still have 893 dblock files, the smallest of which is 14.4 MiB, and the largest of which is 50.1 MiB. There’s a smoother distribution too, and as I’ve just noticed now, it’s that same eerie smoothness that I mentioned before this edit. The only difference is the range is 14 through 50, as opposed to the compact producing 1 through 50. Curious, I think. Or not, now that I use my brain. I suppose any sufficiently large dataset would provide opportunities for dblock files of all those sizes, really. I’m bad at math and know nothing meaningful of statistics, so take my observations with a grain of salt.

[kenkendk]

@ItEndsWithTens

Building the tray icon executable from the latest code and debugging that, however, then following the repro steps from my previous post (not reusing any of my old existing backups, or previous test configs, but starting fresh), I see that by the same point, the compression-module key is already present, and set to “zip”, so it remains untouched and the problem persists.

If I modify that unit test to also add a “compression-module” key with the value “zip”, right alongside “no-encryption” when initializing that Dictionary, the unit test fails, and the recreated database retains the zip compression setting.

That makes sense. If the user provides the option (via the Dictionary), there is (likely) a good reason for it, so we do not override with anything autodetected. We can discuss if that is a good logic for the recreate step. I think we should perhaps fail if the backup contains mixed compression or does not match the given --compression-method.

In your steps, I think that step (5), needs to be:

• 5a: Back in the web GUI, edit backup and set compression module to tzstd
• 5b: Recreate the database deleted in step 3.

If you do not do this, you are building a new database and explicitly asking it to use zip as the compression method, and this will be baked into the database. What I fixed was a related issue where the input has no --compression-module set, and the code would then assume that zip should be used.

I assumed both were the same case, but I can see now that they are different.

@ItEndsWithTens

it ballooned the file count from 1,656 files before, to 3,534 after.

That was not expected. It should generally just produce the same output as before, but with compression (and encryption) tiny variations can cause large swings.

Not sure how we can reliably detect the current size of the output. We could build the file concurrently with the processing, but since we cannot hook into the compression library, we cannot know how much data is buffered (and not flushed to disk), so observing the file size on disk really tells us nothing.

[ItEndsWithTens]

@kenkendk

In your steps, I think that step (5), needs to be:

• 5a: Back in the web GUI, edit backup and set compression module to tzstd
• 5b: Recreate the database deleted in step 3.

May I ask what the intended method is to accomplish step 5a? Because I’d tried something like this in the days leading up to my initial report, but ran into trouble.

Using ngclient, I follow my steps up through number 4, then edit the backup configuration. There is no visible way to change the compression type on the “General” page, so I go to “Options” to add an advanced option. Clicking “Add advanced option” gives me a drop down list of available choices, but none of them are ‘compression-module’. There is ‘zip-compression-method’, but as I understand that’s just for zip files.

Because that option isn’t visible, I click the three dot menu and choose “Edit contents as text”, and add --compression-module=tzstd at the bottom, then click the blue Submit button to save the backup.

I recreate the database, then examine it in DB Browser for SQLite to confirm the change has taken effect, and indeed it has. The local database for this backup has, in its Configuration table, the ‘compression-module’ key set to ‘tzstd’.

What I notice that’s odd, and that I believe caused the “trouble” I mentioned, is that:

• When subsequently editing the backup configuration, the --compression-module=tzstd advanced option I added manually has vanished. It’s not shown in the graphical editor, nor in the “edit as text” field.

• Going to the Command Line for this backup shows, in the Advanced options at the bottom, two instances of “Select what module to use for compression”, the first being zip and the second tzstd. This appears to originate in the Duplicati-server.sqlite database file, as looking in the Option table for the relevant BackupID shows both ‘compression-module’ and ‘–compression-module’ keys, one without and one with the em dash prefix, with the former set to ‘zip’ and the latter to ‘tzstd’.

This doesn’t appear to cause problems at first, for a small backup like this test example, but before initially replying to this thread I was trying it on a few real backups that were larger and had space that could be compacted. Attempting to run Compact on the small example backup finishes without error since there’s no compaction to perform, but trying it on my real backups caused errors about attempting to change the compression type, if I remember correctly.

[kenkendk]

@ItEndsWithTens

Using ngclient, I follow my steps up through number 4, then edit the backup configuration. There is no visible way to change the compression type on the “General” page, so I go to “Options” to add an advanced option. Clicking “Add advanced option” gives me a drop down list of available choices, but none of them are ‘compression-module’. There is ‘zip-compression-method’, but as I understand that’s just for zip files.

Because that option isn’t visible, I click the three dot menu and choose “Edit contents as text”, and add --compression-module=tzstd at the bottom, then click the blue Submit button to save the backup.

Yes, I can confirm this is a problem. The issue is that we used to have “compression module” prominently so we hide/remove it from the options list. Then at some point we simplified the UI as there was only a single compression method, so now the UI simply removes any attempt to set that option.

[ItEndsWithTens]

Ah, that explains it then, thank you! I mistakenly made assumptions based on that behavior in the web GUI, as well as this passage from the recovery tool docs:

Warning: Before recompress delete the local database and after recompress recreate local database before executing any operation on backup. This allows Duplicati to read new file names from remote storage.

With no mention of manually changing compression type in the backup config, I thought the only way to set the compression by hand would be at the initial configuration of a brand new backup, and that when using the recovery tool to recompress you’d have to let Duplicati detect a change in compression automatically. But of course it had slipped my mind that the only recompression one could do until recently was change encryption, or other such characteristics that would still result in Deflate-formatted .zip files. Perhaps a silly inference on my part, since the quoted help text only mentions file names, not compression type.

[ts678]

@kenkendk

Added remote-volume-size option alias

How is this supposed to be used? It doesn’t seem to be on any dropdown.

Remote volume size and dblock-size removed? #456 gave context for this.
Current situation is that legacy UI complains about no remote volume size.
Maybe the answer is ignore that? At least it looks like dblock-size saves:

[ItEndsWithTens]

Just want to confirm that with the release of Canary 2.3.0.103 I can follow my repro steps successfully now. Switching the compression type with the convenient drop down list before rebuilding the database works like a charm, the local and server databases end up with the right values for compression-module with no duplicate entries, and the example purge command does what I expect. Very well done, thank you!