Release: 2.1.1.0 (Experimental) 2025-07-17

2.1.1.0_experimental_2025-07-17

This release is an experimental release inteded to be used for testing.

The intention is to adjust any issues uncovered with this version, and eventually get a new stable release.

About this release

This is the first experimental release since the stable release went out, and we are super excited to reach this milestone!

The most visual change for this version is the use of the new user interface, but there is also a massive list of fixes and improvements in this version. Below is a summary of some of the larger changes.

New user interface

The new user interface is rewritten from scratch and has the same general structure as the previous one, but we made some things more user friendly. The UI is fully functional, but we continue to improve on it.

Should you find a function that is missing, we have included buttons to switch back-n-forth between the two user interfaces.

New backends

We added support for using the cloud services pCloud, Filen and Filejump.

We also added support for connections with SMB.
The new SMB backend can connect directly to a Windows share without needing to mount the folder or install SMB support, and it works on Windows, Linux and macOS.

New restore flow

The new restore flow is enabled by default and you should not notice anything other than faster restores. In case there is an issue with this, it is possible to set the option --restore-legacy=true to fall back to the previous restore flow.

New signing keys

The packages are now signed by Duplicati Inc, and the Windows packages are signed with EV certificates.

Remote source support

With this version it is now possible to make backups of local and some remote data.

In this version, S3, IDrive, SSH and CIFS sources are supported.
The UI does not yet support editing this nicely, but you can enter a path in the special format to “mount” the remote source.

For the commandline (and manual text entry in the UI) enter sources such as:

// Linux/MacOS
@/mnt/s3-data|s3://example?auth-username=...

// Windows
@X:\server1|smb://server/share?auth-username=...

This will cause the backups to fetch data from the remote sources.
We will add an editor to the UI to allow browsing the remote sources, similar to the local files.

Archive attributes support

For AWS S3 and Azure Blob Storage, Duplicati will now respect the archive attributes and not attempt to read and verify files that have been moved to cold storage.

Database updates

This version updates the format of the local database to version 17.

To assist in downgrades there is now a bundled CommandLine.DatabaseTool.exe / duplicati-database-tool that can downgrade databases with minimal data loss. For a downgrade from this version to 2.1.0.5 this will only drop a few indexes and not cause any data loss. Be sure to run the database tool before downgrading the install.

Throttle updated

For backups that throttle the transfer speeds, the new throttle logic uses a shared limit for the backup, where previous versions would apply the throttle for each individual stream.

Removed backends

The Sia backend has been removed due to an incompatible hardfork.

The Mega backend has been marked as unmaintained due to lack of a supported library.
For now, the Mega library still works, but you should migrate away from it. The new Mega S4 storage might be an option.

Updates to all backends

All backends are updated to handle timeouts in a granular manner.

This means the option --http-operations-timeout is no longer present, but instead there are now --read-write-timeout, --list-timeout, and -short-timeout. These have sensible defaults but are open for tweaking.

The option --allowed-ssl-versions is only present for the FTP backend, all other backends use the operating system to figure out what version to use.

New datafolder default location

For Duplicati running as a service there are now changes for the default folder location.
If you are not running Duplicati as a service/daemon, this change has no effect.

Windows: Avoid storing data in C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati and prefer {CommonProgramData}\Duplicati, usually resolving to C:\ProgramData\Duplicati.

This change is to counter an issue where Windows will wipe the C:\Windows folder on major updates, and destroy the backup configuration in the process. If your service stores data under C:\Windows you will see a warning in the user interface on startup.

Linux: Avoid storing data in /Duplicati and prefer /var/lib/Duplicati.
This was caused by the update to .NET8 where the data folder was not resolved correctly and returned /, which is not the ideal place for storing data.

If you are using --server-datafolder or DUPLICATI_HOME, this has no effect on the database, but may cause your machineid and installid to change.

The machineid.txt and installid.txt would previously be stored in the local app data folder, even when using portable mode or choosing a specific data folder.

This has been fixed, so the files will now follow the database.
If you are using the Duplicati console or otherwise depend on these values, you need to move them into the folder where the database is stored.

This update also sets permissions on the data folder and the databases to prevent unauthorized access from local accounts.
To opt out of setting permissions on each startup, place a file named insecure-permissions.txt inside the data folder.

Other large changes

  • New file and folder enumeration logic
  • Timeouts logic on all backend operations
  • Improved database validation and repair logic
  • ServerUtil can output JSON for script integration
  • Improved support for having Duplicati behind a proxy
  • Updated throttle logic, all streams share the throttle
  • Improved repair logic
  • VSS is automatically on if running on Windows with sufficient privileges
  • Improved backend test function
  • Ability to suppress warnings
4 Likes

I’ve been testing this version for a few days and thus far it seems to be working well, except for one thing.

It frequently reports that it found faulty index files and that it has repaired them:

"2025-07-22 18:06:24 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 1 faulty index files, repairing now"

I have run the Test operation with “all and --full-remote-verification=true” options in an attempt to just have it repair the faulty indices, but it never seems to actually fix them all… unless of course this isn’t the way to actually fix them.

One other problem I have run into is that the ngclient seems to log me out after a short period of time, which the old client never did. When it happens, it seems to go into this weird loop where it flickers between the “Log In / Out” screens. Here’s a GIF of it … sorry for the poor quality, it’s the best the Windows screen recorder can manage:

2025-07-22-duplicati

Mine acts a little different, but ngclient does need refresh a lot to get back to connected.
Improve re-connect handling #283 is an ngclient issue, maybe with some progress on it.
Maybe yours is different? Mine isn’t exactly logout, and most often it happens overnight.
For yours, sometimes web developer tools (often on F12) are helpful, to see the activity.

While I’m here I’ll ask dev what Experimental is in terms of Canary releases. Changelog suggests it’s based on 2.1.0.120_canary_2025-06-24, but it might have some later fixes.
One thing it doesn’t have is Microsoft.Data.Sqlite.

EDIT 1:

Mine is a countdown popup, generally after a morning computer wake. Today it’s like this:

with the countdown (now at 0:12) cycling back to 0:15 with nothing special on network (on right side of image) when it cycles. The websocket activity cycles at every 4 or 5 seconds. Browser refresh gets it connected without other manual work, and network winds up quiet.

I think this is the new “Remember me” feature in play. In all but the most recent builds, there is no “Remember me” button, so logging in will keep you logged in if you check the backup at least once every 6 months.

Now that the checkbox is there, it will default to not keeping you logged in. This works by creating a short-lived (30 min) token that the browser keeps. Once the token has expired, you will be logged out, but it looks like it does not correctly clear this expired token and repeatedly tries to log in with it.

I have an idea for improving this feature with improved security.

This seems to indicate that the faulty index files are actually being created? Can you somehow see the timestamp on any of the files that are detected as a faulty? (difficult perhaps, as they are deleted after being repaired).

If it’s the access token, I think it’s 15 minutes without extension by user activity. Crude.
Switch to old UI gets logoff regardless of time since new UI login. Is that as expected?

I agree. I have fixed it in canary.

No, the ngax client does not have the concept of non-persistent login, so it should work the same as before, where a refresh token is stored as a cookie and will grant a new 15 minutes access if used within 90 days.

Maybe you’re thinking this is all ngax? I meant login to new ngclient UI without “Keep me logged in”. Switch to old UI gets error. It doesn’t matter whether or not these steps begin with ngax login, however maybe if ngax lacks the concept of non-persistent login it can’t fathom the ngclient login without the checkbox checked, which was said to be default but for me it remembers past choice.

My backups were working fine (for years) up through 2.1.0.1_experimental_2024-11-25 but I started getting “volumes with missing filesets” errors with every run after installation of 2.1.1.0_experimental_2025-07-17.

Has anybody else run into this? I’m trying to figure out if it’s possibly a coincidence, an improvement in error detection finding something that’s been wrong for a long time, or an upgrade-introduced error.

Yes, this is the case.

Yes, ngclient remembers the last choice with a local-storage value. This has no effect on ngax.

The check has been there for quite some time, it is present in 2.1.0.5 as well.

I assume you upgraded from 2.1.0.1 to 2.1.1.0 ? If so, then the check is new, and this is an improved error detection (with repair code).

Yep - upgraded from 2.1.0.1. Thanks for confirming it’s improved detection!

A 10 minute database repair resolved the “compressed using an unsupported compression method” issue on one of my backups for me.

The straight “Repair” option but didn’t seem to help with backup having the “volumes with missing filesets” problem, unfortunately a 1 hr “Delete and Repair” (on a 488M DB) also didn’t resolve it.

My TB backup on 2.1.2.0 Beta seems to make faulty index files over repeat test all.

2025-09-12 21:20:38 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 106 faulty index files, repairing now
2025-09-13 15:33:29 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 4 faulty index files, repairing now
2025-09-13 23:17:26 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 4 faulty index files, repairing now

It did do a good job of cutting down my initial faulty ones, so I figured I’d run it again, and was surprised that it still had 4 it didn’t like. I ran it again, got another 4, and began study.

The problem seems to be in the case where a block is in multiple dblock files (which can happen due to assigning a new block instead of recycling block from DeletedBlockTable).

With a duplicate destination block, which dindex (or both) is supposed to claim the block?

For example, Uploading file duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip is replacement dindex of duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip having block /UGSPS1hNl7pDhzFh2PJmgVy02faZO1h/v7zY85kf3s=, but omits indexing that block. Block is registered in DB to duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip, but DuplicateBlock volume is duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip.

New dindex might just index what’s registered with its dblock, which here is a block short, however I’m not familiar enough with FaultyIndexFiles detection to say if that will trigger it.

EDIT 1:

Here’s Duplicati calling its new dindex files faulty, but is the file faulty, or is the test faulty?

2025-09-13 15:33:29 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 4 faulty index files, repairing now
2025-09-13 15:34:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 15:34:27 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 15:34:28 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i5ed798c4c6fb49e9be82cbd7fe20b56f.dindex.zip (351.52 KiB)
2025-09-13 15:34:28 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i5ed798c4c6fb49e9be82cbd7fe20b56f.dindex.zip (351.52 KiB)
2025-09-13 15:34:33 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 15:34:33 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 15:34:33 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ia2a79da5cb19416ea25341e08a385d77.dindex.zip (453.93 KiB)
2025-09-13 15:34:33 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ia2a79da5cb19416ea25341e08a385d77.dindex.zip (453.93 KiB)
2025-09-13 15:34:38 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 15:34:38 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 15:34:38 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-iff5458e1617a4d85a38388b31d443c6c.dindex.zip (182.62 KiB)
2025-09-13 15:34:38 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-iff5458e1617a4d85a38388b31d443c6c.dindex.zip (182.62 KiB)
2025-09-13 15:34:43 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)
2025-09-13 15:34:43 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)
2025-09-13 15:34:43 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i16af17c7c7c04582bd7a8b83d5a0a39b.dindex.zip (99.64 KiB)
2025-09-13 15:34:43 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i16af17c7c7c04582bd7a8b83d5a0a39b.dindex.zip (99.64 KiB)
2025-09-13 15:34:45 -04 - [Information-Duplicati.Library.Main.Operation.TestHandler-Test results]: Successfully verified 3036 remote files
2025-09-13 15:34:48 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Test has completed
2025-09-13 20:01:03 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Test has started
...
2025-09-13 23:17:26 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 4 faulty index files, repairing now
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip (351.52 KiB)
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip (351.52 KiB)
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-iba1200144eee47b18960aec13303b92a.dindex.zip (453.94 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-iba1200144eee47b18960aec13303b92a.dindex.zip (453.94 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i48e3cddf08804e3785b8444fcff65b9a.dindex.zip (182.62 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i48e3cddf08804e3785b8444fcff65b9a.dindex.zip (182.62 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i5fd6d2d01b984295ad4b1a7db05110a4.dindex.zip (99.64 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i5fd6d2d01b984295ad4b1a7db05110a4.dindex.zip (99.64 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)
2025-09-13 23:17:47 -04 - [Information-Duplicati.Library.Main.Operation.TestHandler-Test results]: Successfully verified 3036 remote files
2025-09-13 23:17:50 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Test has completed

Here is the DuplicateBlock table which is suspected to be part of the special failure mode:

BlockID VolumeID
3126635 2226
3849103 2549
3840883 2549
3849096 2549
3840884 2549
3840885 2553
3840882 2553
3849107 2575
3846018 2575
3852986 2593

That’s 10 rows, but the first one references an invalid BlockID. Translating others by SQL:

SELECT Block.Hash, RemoteVolume.Name
FROM DuplicateBlock
JOIN Block ON BlockID = Block.ID
JOIN RemoteVolume ON DuplicateBlock.VolumeID = RemoteVolume.ID
Hash Name
EakUKlcNjXGt0z+WGzHiPgECIPYIj59K3k1DsGMSkPU= duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip
TubCF35n0nwNFLCwMTfj7+olnxWzvFp5xav8W8/UlD4= duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip
Ubi9P6rs3wkSLzvBVGQGuBBrvufAhifV5/ZjzZxhXss= duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip
0kxgm/emmIESLjJws4Pve9bCQVY6AwJMvYNdzxoUzpA= duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip
8FOLkQcwVtkvSrm4TQzVAhZcI5nuvcMpk9YuhlWAAKU= duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip
I5sH1OvTeVSi+vnRR4hUMFZOz133PE9ATDj+8Co70wg= duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip
28RKR+16B7ALvZvFmJVG9df86If+HGi6pmBjODpDpLQ= duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip
qQut45YDw2wEgw6aSqlbiHlkrDi0n61e8MCRhzUp59k= duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip
/UGSPS1hNl7pDhzFh2PJmgVy02faZO1h/v7zY85kf3s= duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip

Actually looking into the dblock files’ content with a Python script finds 11 dups, it appears:

Duplicate block aAN3V-ybscgo5ZwPKJSddqWH-ow4oeHVCj9P0brGjEU= in duplicati-b7fe27bc0b1644b5eaac9979886b4c2cb.dblock.zip first seen in duplicati-bf95bb947e5b143e1a7cb1f7c8a8f2205.dblock.zip
Duplicate block EakUKlcNjXGt0z-WGzHiPgECIPYIj59K3k1DsGMSkPU= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Duplicate block TubCF35n0nwNFLCwMTfj7-olnxWzvFp5xav8W8_UlD4= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
Duplicate block Ubi9P6rs3wkSLzvBVGQGuBBrvufAhifV5_ZjzZxhXss= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Duplicate block 0kxgm_emmIESLjJws4Pve9bCQVY6AwJMvYNdzxoUzpA= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
Duplicate block 8FOLkQcwVtkvSrm4TQzVAhZcI5nuvcMpk9YuhlWAAKU= in duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
Duplicate block I5sH1OvTeVSi-vnRR4hUMFZOz133PE9ATDj-8Co70wg= in duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
Duplicate block 28RKR-16B7ALvZvFmJVG9df86If-HGi6pmBjODpDpLQ= in duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Duplicate block qQut45YDw2wEgw6aSqlbiHlkrDi0n61e8MCRhzUp59k= in duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Duplicate block _UGSPS1hNl7pDhzFh2PJmgVy02faZO1h_v7zY85kf3s= in duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Duplicate block rPX0nXvuhT2mX9tQ6qaC1_BHkG4QfIrJvSy7g_piRVY= in duplicati-ba9b89b8cf9574b30906d0116e165e4ee.dblock.zip first seen in duplicati-b473ac4ceb393428eba28c4959706d6a0.dblock.zip

EDIT 2:

The two above views are similar, but there’s still 9 rows versus 11. The first line just above looks like the block that got lost track of, causing a line drop from 10 to 9. The last line just above looks like a small blockset, but I’m not yet sure if it’s visible in faulty dindex analysis.

ID Length FullHash
3522972 1731 rPX0nXvuhT2mX9tQ6qaC1/BHkG4QfIrJvSy7g/piRVY=
ID Hash Size VolumeID
4590617 rPX0nXvuhT2mX9tQ6qaC1/BHkG4QfIrJvSy7g/piRVY= 1731 2654

Second faulty dindex diff analysis seems to lead me to

28RKR+16B7ALvZvFmJVG9df86If+HGi6pmBjODpDpLQ=
qQut45YDw2wEgw6aSqlbiHlkrDi0n61e8MCRhzUp59k=

EDIT 3:

I still had the GUI Commandline window open, and intended to copy more, but I got the annoying overnight connection loss. Rather than refresh, I tried duplicating, but it ran…

Near bottom, the line about an Extra: block is probably because Duplicati lost track of it.

EDIT 4:

The duplicated tab isn’t actually running, just trying. Every second it sends this request in:

http://localhost:8200/api/v1/commandline/9145a416-b861-48cc-aea9-c8ba280ce9ab?offset=0&pagesize=100

Server says {"Error":"Command not found","Code":404} and Alerts logs this faster than I can click them away. The Abort button isn’t stopping the flood, so I closed the tab. Opened a new (not duplicated) tab, and it’s quiet and clean. Not even old Alerts. Old was:

EDIT 5:

I wanted some sort of test to see if the flagged index file gave any other negative effects.
On database screen, I changed its name to one with test-recreate for Save & Repair.

2025-09-16 10:41:57 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-i4980c24c548347189a107712ed86391c.dindex.zip (96.38 KiB)
2025-09-16 10:52:40 -04 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RecreateCompletedCheckingDatabase]: Recreate completed, verifying the database consistency
2025-09-16 10:59:54 -04 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RecreateCompleted]: Recreate completed, and consistency checks completed, marking database as complete
2025-09-16 11:03:16 -04 - [Information-Duplicati.Library.Main.Controller-CompletedOperation]: The operation Repair has completed

looks good, so seemingly the perceived faulty index files are not damaging critical things.

I looked in the database from Recreate. FWIW the other database was also by Recreate.
Somehow new DB managed to put 35616 rows in DeletedBlock, just like the old one had.
DuplicateBlock had only two rows, but again the first row somehow got an invalid BlockID.

SQL posted above only had one valid row left, and it expanded into a familiar block hash, visible by the direct dblock analysis, but not present in the previous DuplicateBlock study:

Hash Name
rPX0nXvuhT2mX9tQ6qaC1/BHkG4QfIrJvSy7g/piRVY= duplicati-ba9b89b8cf9574b30906d0116e165e4ee.dblock.zip

EDIT 6:

EDIT 7:

The smaller DuplicateBlock table compared to the direct dblock analysis could be due to dindex index of one of the dblocks not listing the block because DB had it in other dblock.

Base post here before edit gives example, and asks which (or both) dindex should index.

EDIT 8:

Pairing up the direct dblock inspection duplicate list with DuplicateBlock table breakdown:

Duplicate block aAN3V-ybscgo5ZwPKJSddqWH-ow4oeHVCj9P0brGjEU= in duplicati-b7fe27bc0b1644b5eaac9979886b4c2cb.dblock.zip first seen in duplicati-bf95bb947e5b143e1a7cb1f7c8a8f2205.dblock.zip

Duplicate block EakUKlcNjXGt0z-WGzHiPgECIPYIj59K3k1DsGMSkPU= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
EakUKlcNjXGt0z+WGzHiPgECIPYIj59K3k1DsGMSkPU= 	duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip

Duplicate block TubCF35n0nwNFLCwMTfj7-olnxWzvFp5xav8W8_UlD4= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
TubCF35n0nwNFLCwMTfj7+olnxWzvFp5xav8W8/UlD4= 	duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip

Duplicate block Ubi9P6rs3wkSLzvBVGQGuBBrvufAhifV5_ZjzZxhXss= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
Ubi9P6rs3wkSLzvBVGQGuBBrvufAhifV5/ZjzZxhXss= 	duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip

Duplicate block 0kxgm_emmIESLjJws4Pve9bCQVY6AwJMvYNdzxoUzpA= in duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
0kxgm/emmIESLjJws4Pve9bCQVY6AwJMvYNdzxoUzpA= 	duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip

Duplicate block 8FOLkQcwVtkvSrm4TQzVAhZcI5nuvcMpk9YuhlWAAKU= in duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
8FOLkQcwVtkvSrm4TQzVAhZcI5nuvcMpk9YuhlWAAKU= 	duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip

Duplicate block I5sH1OvTeVSi-vnRR4hUMFZOz133PE9ATDj-8Co70wg= in duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip first seen in duplicati-b983ef925f4ec455ba3814c2798c73007.dblock.zip
I5sH1OvTeVSi+vnRR4hUMFZOz133PE9ATDj+8Co70wg= 	duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip

Duplicate block 28RKR-16B7ALvZvFmJVG9df86If-HGi6pmBjODpDpLQ= in duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
28RKR+16B7ALvZvFmJVG9df86If+HGi6pmBjODpDpLQ= 	duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip

Duplicate block qQut45YDw2wEgw6aSqlbiHlkrDi0n61e8MCRhzUp59k= in duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
qQut45YDw2wEgw6aSqlbiHlkrDi0n61e8MCRhzUp59k= 	duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip

Duplicate block _UGSPS1hNl7pDhzFh2PJmgVy02faZO1h_v7zY85kf3s= in duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip first seen in duplicati-b4663926b25db4a5ab2f0bc5a6a433c3c.dblock.zip
/UGSPS1hNl7pDhzFh2PJmgVy02faZO1h/v7zY85kf3s= 	duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip

Duplicate block rPX0nXvuhT2mX9tQ6qaC1_BHkG4QfIrJvSy7g_piRVY= in duplicati-ba9b89b8cf9574b30906d0116e165e4ee.dblock.zip first seen in duplicati-b473ac4ceb393428eba28c4959706d6a0.dblock.zip

This is the database before Recreate, which I’ve commented on above and am back on.

EDIT 9:

Having looked at the dblock situation, translate back to index and insert in original issue:

2025-09-13 23:17:26 -04 - [Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 4 faulty index files, repairing now
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip (351.52 KiB)
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip (351.52 KiB)
duplicati-i69d4968a7f7c4efe81369d021d12db4e.dindex.zip duplicati-b80e6f95c10f4407c8ef1d235adb24382.dblock.zip

2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 23:17:29 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ief3f8c92f4114c809fec147efa75b183.dindex.zip (351.52 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-iba1200144eee47b18960aec13303b92a.dindex.zip (453.94 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-iba1200144eee47b18960aec13303b92a.dindex.zip (453.94 KiB)
duplicati-iba1200144eee47b18960aec13303b92a.dindex.zip duplicati-bf28caf15681e4f32b51330f08529805c.dblock.zip

2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 23:17:34 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i986c48a174694956bc211ad7cda67625.dindex.zip (453.94 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i48e3cddf08804e3785b8444fcff65b9a.dindex.zip (182.62 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i48e3cddf08804e3785b8444fcff65b9a.dindex.zip (182.62 KiB)
duplicati-i48e3cddf08804e3785b8444fcff65b9a.dindex.zip duplicati-ba8153f280fd54761adad5ac56317f449.dblock.zip

2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 23:17:39 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ib08a17cb28f84b21a098917b5000a1ba.dindex.zip (182.62 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i5fd6d2d01b984295ad4b1a7db05110a4.dindex.zip (99.64 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i5fd6d2d01b984295ad4b1a7db05110a4.dindex.zip (99.64 KiB)
duplicati-i5fd6d2d01b984295ad4b1a7db05110a4.dindex.zip duplicati-b60656806a0c04afa8388ce89847a9fa9.dblock.zip

2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)
2025-09-13 23:17:44 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i444640ce087d4ef480e92a17606ac622.dindex.zip (99.64 KiB)