Any way to recover backup (repair and purge-broken-files don't help)?

ts678 · November 22, 2023, 4:27pm

Generally agree, however there are a couple of places where the local database can have some fixes:

duplicati/duplicati/blob/ec9d49d6849b3b37f965ac80be45e1460d21417c/Duplicati/Library/Main/Operation/RepairHandler.cs#L486-L489


      
          db.FixDuplicateMetahash();

          db.FixDuplicateFileentries();

          db.FixDuplicateBlocklistHashes(m_options.Blocksize, m_options.BlockhashSize);

          db.FixMissingBlocklistHashes(m_options.BlockHashAlgorithm, m_options.Blocksize);

Speaking of blocklist hashes, they might be involved in

github.com

duplicati/duplicati/blob/ec9d49d6849b3b37f965ac80be45e1460d21417c/Duplicati/Library/Main/Operation/RepairHandler.cs#L329-L336


      
          foreach (var b in db.GetBlocklists(volumeid, m_options.Blocksize, hashsize))

          {

              var bh = Convert.ToBase64String(h.ComputeHash(b.Item2, 0, b.Item3));

              if (bh != b.Item1)

                  throw new Exception(string.Format("Internal consistency check failed, generated index block has wrong hash, {0} vs {1}", bh, b.Item1));

          

              w.WriteBlocklist(b.Item1, b.Item2, 0, b.Item3);

          }

and after that I got lost in SQL, but it made me wonder if some other internal records were misaligned.
A formatted view of the GetBlocklists query being run was more readable, but still stretching my skills:

SELECT "A"."Hash"
    ,"C"."Hash"
FROM (
    SELECT "BlocklistHash"."BlocksetID"
        ,"Block"."Hash"
        ,*
    FROM "BlocklistHash"
        ,"Block"
    WHERE "BlocklistHash"."Hash" = "Block"."Hash"
        AND "Block"."VolumeID" = 2
    ) A
    ,"BlocksetEntry" B
    ,"Block" C
WHERE "B"."BlocksetID" = "A"."BlocksetID"
    AND "B"."Index" >= ("A"."Index" * 3200)
    AND "B"."Index" < (("A"."Index" + 1) * 3200)
    AND "C"."ID" = "B"."BlockID"
ORDER BY "A"."BlocksetID"
    ,"B"."Index"

If you get into this, please check my doing. I just grabbed a likely-looking query from my profiling log…