Assuming “lazy” means broken in some way that gets complaints, and that complaints are persistent over several runs with “all”, the problem looks to me like the replace-faulty-index-files can make faulty replacement index files that lack blocks from DeletedBlock table (meaning blocks that now site in dblock file but are waste after their use ages away).
It seems to pick up blocks from Block table, which is current, and I’m not sure what to say about DuplicateBlock table which is even more obscure. This info is mainly for developer.
I tested with backup having 7 known bad dindex files (no list folder). I had test sample less than “all”, so it claimed to fix 4 by upload of a new dindex then delete of the old dindex.
Found 4 faulty index files, repairing now
Uploading file duplicati-i63c3448043e54cf686629a88285d85e7.dindex.zip.aes (68.53 KiB) ...
Deleting file duplicati-ibae40f55c39f4bf39688dd5a84431fce.dindex.zip.aes (71.25 KiB) ...
Uploading file duplicati-i8f5c3631ef834479bd40f18afb529c62.dindex.zip.aes (251.08 KiB) ...
Deleting file duplicati-if295f13bb37847de88d2f823afc932b8.dindex.zip.aes (230.09 KiB) ...
Uploading file duplicati-idc1c75b308924e24937e2605aebee027.dindex.zip.aes (44.93 KiB) ...
Deleting file duplicati-i2ea29e9ab8364952872e2eff6fa5c0af.dindex.zip.aes (33.56 KiB) ...
Uploading file duplicati-ic6a2f0e4dd264b239792ea8363bf671e.dindex.zip.aes (53.01 KiB) ...
Deleting file duplicati-ia2516d45091e4471bd171f77bebee323.dindex.zip.aes (22.98 KiB) ...
I tested all, expecting 7 - 4 = 3 bad dindex files, but got 5. 2 of 4 are still flagged:
duplicati-i63c3448043e54cf686629a88285d85e7.dindex.zip.aes: 849 errors
duplicati-i8f5c3631ef834479bd40f18afb529c62.dindex.zip.aes: 41 errors
duplicati-idc1c75b308924e24937e2605aebee027.dindex.zip.aes: No errors
duplicati-ic6a2f0e4dd264b239792ea8363bf671e.dindex.zip.aes: No errors
Looking at files that got replaced by 2 new bad ones, they respectively looked like:
duplicati-ibae40f55c39f4bf39688dd5a84431fce.dindex.zip.aes: 21 errors
duplicati-if295f13bb37847de88d2f823afc932b8.dindex.zip.aes: 65 errors
Some of this might be from passage of time helping to increase the wasted space issue, however superficially numerically, the problem in the first file grew far worse than before.
Actual impact might not be worse, as either one might require reading associated dblock. Developer comment on the theory and its impact would help, but there’s proposed cause.
EDIT 1:
Rather than sample more of the missing blocks to see where I could find them, I looked in DeletedBlock table for row count in the dblock associated with the new dindex. Result is:
duplicati-b1d9a404a6302440d820be51e620975e0.dblock.zip.aes has 849
duplicati-b08fe3c79b7e54033bc0fab0225ae14ee.dblock.zip.aes has 41
thereby matching the missing blocks in dindex, and supporting the DeletedBlock theory.
I’m tempted to delete the bad dindex to see what repair does when it replaces the dindex. Seeing it do the same error would mean there’s another route to bad dindex. I’ll wait a bit.
EDIT 2:
While leaving my larger backup alone, I tested a small one which unfortunately looks like Repair makes the same error, and even worse it makes it on 2.1.0.5 Stable. Steps to test:
- Make unencrypted backup, keeping 1 version, of A.txt and B.txt each with that letter.
- Deselect B.txt and backup again. Looking in the DB, this splits the original situation where A.txt and B.txt each had a data block and a metadata block in the Block table.
- Delete or hide (e.g. change prefix) the dindex file, and Database Repair to replace it.
- Extract the vol file from new dindex and notice it only has two blocks rather than four.
{"blocks":[{"hash":"VZrq0IJk1XldOQlxjN0Fq9SVcuhP5VWQ7vMaiKCP3/0=","size":1},{"hash":"UW7CCxtLGw1gqSW6qxiGU8Xd/arQWsoW55bRZeAeQug=","size":137}],"volumehash":"LrqTo3JmaqseCgJKz52Te2L5VJt++iJR0FPzmrVNJzQ=","volumesize":1128}
The dblock file has four blocks, therefore the dindex file associated with it should index all.
EDIT 3:
To clarify “splits” and show what went where, here are Block and DeletedBlock tables.
As had been suspected, blocks for that volume are lost from dindex if in DeletedBlock, however let’s see if that actually causes a dblock read, as the dlist reference is gone too.
Database Recreate did not read the dblock, and DeletedBlock table was not populated.
Running test all with full-remote-verification complains of extra dblock blocks:
duplicati-b065a67d0d8574c9587ced93727e109a4.dblock.zip: 2 errors
Extra: 335w5QIVRPSDS77mSp43if68S+gUcN9inK1t2wMyClw=
Extra: h90JSQPN+uqg6ows41QAIdBiEbLFXeNVDrx740zJLVE=
So the good news is it doesn’t seem to have slowed a DB recreate, but is this still a bug?
In addition to new “Extra” case, there’s also the “Missing” ones that were reported earlier.