How to recover backup with damaged dblock file?

Support
1

I have a backup which somehow got partially broken at some point – specifically it appears that one of dblock files is damaged. This is a local backup (a folder on one of my disks).

If I run verify with --full-remote-verification=true I get errors like this:

duplicati-ie747e641fbf94c1cb70f9c004b20f069.dindex.zip.aes: 365 errors
	Extra: +15x4rhrFdJDxsq7//w9oOIxTrTwGjcTfhLxI2/LJyA=
	Extra: +1YAqb8XnyIvNqcezl76c3thVrOCN9Df4o6bgkPeYBA=
	Extra: +AmEoB4m0pAoEV0jChwT6n9a52oNZcbYpOhtKBbdGAY=
	Extra: +I/t35312puVMY8o3is6t+ZTYnx+gh+bl5YMwyN5nIw=
	Extra: +mGEnIsjOuxxFc6/gDGCr8PP6jTMi1wPj6b5333xM+M=
	Extra: +rkn2PuiFFAKGdVOfw8oi7275ERJGjdIsxDpqMnT0jI=
	Extra: +tiWUyio2o8zvld02ygmD/zPt+7unLFJq139kY0CRCw=
	Extra: +zkHGQSKU3bt9faKtFRI7C3YPiSb/CSjg7FJjeEROgc=
	Extra: /0Dg0RtO4rUoFcFIuXAN49ej9DdguiOSSQ8o+o2+6us=
	Extra: /9lL0SoJpDZvA0x3rOZaOyns8TjKNISv1mKj2T6ir9s=
	... and 355 more


duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes: 1 errors
	Error: Failed to decrypt data (invalid passphrase?): Message has been altered, do not trust content


duplicati-bd2d9cef255244e35a1bfe1f1d8193619.dblock.zip.aes: 365 errors
	Extra: +15x4rhrFdJDxsq7//w9oOIxTrTwGjcTfhLxI2/LJyA=
	Extra: +1YAqb8XnyIvNqcezl76c3thVrOCN9Df4o6bgkPeYBA=
	Extra: +AmEoB4m0pAoEV0jChwT6n9a52oNZcbYpOhtKBbdGAY=
	Extra: +I/t35312puVMY8o3is6t+ZTYnx+gh+bl5YMwyN5nIw=
	Extra: +mGEnIsjOuxxFc6/gDGCr8PP6jTMi1wPj6b5333xM+M=
	Extra: +rkn2PuiFFAKGdVOfw8oi7275ERJGjdIsxDpqMnT0jI=
	Extra: +tiWUyio2o8zvld02ygmD/zPt+7unLFJq139kY0CRCw=
	Extra: +zkHGQSKU3bt9faKtFRI7C3YPiSb/CSjg7FJjeEROgc=
	Extra: /0Dg0RtO4rUoFcFIuXAN49ej9DdguiOSSQ8o+o2+6us=
	Extra: /9lL0SoJpDZvA0x3rOZaOyns8TjKNISv1mKj2T6ir9s=
	... and 355 more


Return code: 3

When I try to restore the latest version, it restores all files but also gives an error:

2018-11-16 18:37:18 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-RemoteFileProcessingFailed]: Failed to process file duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes
System.Security.Cryptography.CryptographicException: Failed to decrypt data (invalid passphrase?): Message has been altered, do not trust content ---> SharpAESCrypt.SharpAESCrypt+HashMismatchException: Message has been altered, do not trust content
   at SharpAESCrypt.SharpAESCrypt.Read(Byte[] buffer, Int32 offset, Int32 count)
   at Duplicati.Library.Utility.Utility.CopyStream(Stream source, Stream target, Boolean tryRewindSource, Byte[] buf)
   at Duplicati.Library.Encryption.EncryptionBase.Decrypt(Stream input, Stream output)
   --- End of inner exception stack trace ---
   at Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   at Duplicati.Library.Main.Operation.TestHandler.DoRun(Int64 samples, LocalTestDatabase db, BackendManager backend)

The files appear to restore fine – I did byte-by-byte comparison against the current files.

I’ve read the disaster recovery manual, however when I run list-broken-files with --full-result I get nothing:

Running commandline entry
Finished!
            
  Listing remote folder ...
Return code: 0

The bigger problem is that when backup is performed, it checks random file(s) and it is currently stuck on the broken file – i.e. on each backup attempt it report an ‘error’ because it cannot read this particular file. Oh, and I did check with SharpAESCrypt.exe – it also gives the same error, so it looks like the file is indeed damaged.

So the question is – how do I repair this backup so that there are no more errors when backing up and/or doing full backup verification?

I tried deleting the bad dblock file and rebuilding the database – after I did that and tried a restore, duplicati reported an error about file numbers mismatch and failed to restore some of the files – so deleting the bad dblock file makes it worse.

What is the correct thing to do in this situation?

Thanks!

2

I’ve seen the “Extra” errors before and if I recall correctly they’re actually warnings that the dblock file has MORE data than expected - I assume that’s why your restore was successful.

Deleting a dblock (where your actual backup data is stored) is pretty aggressive step and not something I would have recommended - I don’t supposed you actually just renamed or moved the dblock file so can bring it back?

The “Message has been altered, do not trust content” error implies the file has been changed such that the decrypted hashes no longer match - so it’s still decryptable, but what comes out doesn’t exactly match what went in.

This could be caused by things like file damage (disk crash?), malicious intent, or even bad memory - but since this is all local to your machine I’m guessing the first is most likely.

Unfortunately, with the dblock file gone there’s nothing much that can be done other than purge the info related to the deleted dblock file.

I’d suggest trying the list-broken-files command again as now the the dblock file is missing it should find some broken items.

3

Sorry for not being very clear in OP.

I did/do have backup copy of the dblock file I’ve tried deleting. In fact I’ve restored it back to the backup folder afterwards – which leaves me in a situation where I can do a restore, but every backup complains about the file (at the end of backup when doing integrity checking).

The problem I have is that list-broken-files shows me nothing – even when I deleted the bad dblock file it still showed me nothing as far as I remember. If you want to, I can give it another shot?

But in general – I need some advice on how to deal with such damaged archives – I accept that there might be some data loss due to corruption, but I need a way to ‘go forward’ – so that backups work going forward with no errors and I am able to restore stuff that was not ‘lost’ originally.

4

Please try list-broken-files again while the dblock is “missing” - if a missing dblock file doesn’t produce a list then there’s definitely a functional bug in Duplicati.

5

So gave it a shot again – with all files in place I’ve run list-broken-files with two options:
–full-result
–log-level=verbose
(because running with just --full-result gave me even less information)

I got this output:

Running commandline entry
Finished!

            
The option log-level is deprecated: Use the log-file-log-level and console-log-level options instead
The operation ListBrokenFiles has started
No broken filesets found in database, checking for missing remote files
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (82 bytes)
Skipping operation because no files were found to be missing, and no filesets were recorded as broken.
Return code: 0

Then I deleted (moved) the ‘bad’ file, recreated the database and the ‘recreate’ process reported an error, here’s the entire output:

MainOperation: Repair

RecreateDatabaseResults:

    MainOperation: Repair

    ParsedResult: Error

    Version: 2.0.3.11 (2.0.3.11_canary_2018-09-05)

    EndTime: 11/19/2018 9:06:18 PM (1542675978)

    BeginTime: 11/19/2018 9:05:19 PM (1542675919)

    Duration: 00:00:59.1585122

    Messages: [

        2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started,

        2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),

        2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (81 bytes),

        2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 21 filelists,

        2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20181013T152522Z.dlist.zip.aes (286.04 KB),

...

    ]

    Warnings: [

        2018-11-19 21:05:34 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes,

        2018-11-19 21:06:18 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes

    ]

    Errors: [

        2018-11-19 21:05:33 -04 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes by duplicati-i5c5d04011e514af8be947f4c66e3d61b.dindex.zip.aes, but not found in list, registering a missing remote file

    ]

    BackendStatistics:

        RemoteCalls: 66

        BytesUploaded: 0

        BytesDownloaded: 2632633021

        FilesUploaded: 0

        FilesDownloaded: 65

        FilesDeleted: 0

        FoldersCreated: 0

        RetryAttempts: 0

        UnknownFileSize: 0

        UnknownFileCount: 0

        KnownFileCount: 0

        KnownFileSize: 0

        LastBackupDate: 1/1/0001 12:00:00 AM (-62135582400)

        BackupListCount: 0

        TotalQuotaSpace: 0

        FreeQuotaSpace: 0

        AssignedQuotaSpace: 0

        ReportedQuotaError: False

        ReportedQuotaWarning: False

        ParsedResult: Error

        Version: 2.0.3.11 (2.0.3.11_canary_2018-09-05)

        Messages: [

            2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started,

            2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),

            2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (81 bytes),

            2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 21 filelists,

            2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20181013T152522Z.dlist.zip.aes (286.04 KB),

...

        ]

        Warnings: [

            2018-11-19 21:05:34 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes,

            2018-11-19 21:06:18 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes

        ]

        Errors: [

            2018-11-19 21:05:33 -04 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes by duplicati-i5c5d04011e514af8be947f4c66e3d61b.dindex.zip.aes, but not found in list, registering a missing remote file

        ]

ParsedResult: Error

Version: 2.0.3.11 (2.0.3.11_canary_2018-09-05)

EndTime: 11/19/2018 9:06:18 PM (1542675978)

BeginTime: 11/19/2018 9:05:19 PM (1542675919)

Duration: 00:00:59.2500238

Messages: [

    2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Repair has started,

    2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),

    2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (81 bytes),

    2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-RebuildStarted]: Rebuild database started, downloading 21 filelists,

    2018-11-19 21:05:19 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20181013T152522Z.dlist.zip.aes (286.04 KB),

...

]

Warnings: [

    2018-11-19 21:05:34 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes,

    2018-11-19 21:06:18 -04 - [Warning-Duplicati.Library.Main.Database.LocalRecreateDatabase-MissingVolumesDetected]: Found 1 missing volumes; attempting to replace blocks from existing volumes

]

Errors: [

    2018-11-19 21:05:33 -04 - [Error-Duplicati.Library.Main.Operation.RecreateDatabaseHandler-MissingFileDetected]: Remote file referenced as duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes by duplicati-i5c5d04011e514af8be947f4c66e3d61b.dindex.zip.aes, but not found in list, registering a missing remote file

]

After that I’ve ran list-broken-files again with the same two options and got:

Running commandline entry
Finished!

            
The option log-level is deprecated: Use the log-file-log-level and console-log-level options instead
The operation ListBrokenFiles has started
No broken filesets found in database, checking for missing remote files
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (81 bytes)
removing file listed as Temporary: duplicati-ba993f4df6a4f4ad3a9f2cfa5e4fd1f63.dblock.zip.aes
Skipping operation because no files were found to be missing, and no filesets were recorded as broken.
Return code: 0

There’s now an additional ‘removing file listed as Temporary’ line – but this is the only thing.

So what could be the next steps?

6

I’ve had a similar problem, with the difference that in my situation Duplicati complained about missing DBLOCK files.

If the backup fails with error messages about (no) existence of files at the backend, I guess that is has something to do with invalid references in one or more DINDEX files.

For me, the solution was to delete all DINDEX files, delete the local database and recreate the local database.

See this thread for more information.

7

That should work. I have not had to do this myself.

I think what happens is that some dindex file is referencing the broken/missing dblock file, which is why it appears as “temporary”. The recreate currently has an issue with trusting the dindex to not have more content than it should, and this sometimes leads to problems where it tries to reference stuff that is not required.

I suggest that you try to remove the broken file, and then run purge-broken-files. It should then discover the missing dblock file and purge all references to it, rewriting files as required. You can try adding --dry-run first, to see what it will do.

8

It sounds like --list-broken-files didn’t see the missing file, does --purge-broken-files use a different file check process?

9

So I’ve experimented with this backup some more and I’m afraid there’s no ‘good’ news.

list-broken-files / purge-broken-files appear to do nothing if I leave dindex files intact – regardless of whether I remove the broken file manually.

Deleting dindex files, deleting the broken file, and rebuilding the entire database sort-of works – it says there are broken files and this time the list/purge commands actually work.

However if I fully test archive afterwards, I still get 365 (or so) ‘Extra’ errors – not sure what this affects.

BUT the worst part is that the restore from the ‘repaired’ archive fails to restore a number of files that it can correctly restore from the ‘un-repaired’ archive – in other ways ‘repairing’ archive makes things significantly worse in some ways. Which doesn’t really make much sense to me as Duplicati appears to be unable to read the ‘broken’ backup file anyway…

10

Apologies for bumping my own thread – but I need to understand – is there anything else that can be done?

Or should I expect that any damaged archive file makes archive pretty much unusable with the only real option is to re-start archive from the start?

11

Sorry I missed this - real life got a bit busy for a while.

Where you able to move on from this issue or is it still around?

I suspect something CAN be done, but it’s not anything I’ve personally dealt with yet so don’t have any direct suggestions. Mostly I’d have to simulate your scenario and test some potential resolutions to figure out what might help. (And to be honest, I can’t promise that would happen very quickly considering the current time of year.)

12

No, I wasn’t able to recover the backup.

I gave up and made a copy of files from another destination (I had the same stuff being backed up to multiple destinations).

My current estimation is that Duplicati is very unreliable and is hard to recommend for any ‘serious’ backup purpose.

However I probably will still continue to try to use Duplicati – it has an enormous advantages that if I find a configuration that more or less works for me, I am not at risk of my setup breaking because someone decided that they must update their app / infrastructure (I’m looking at you, CrashPlan!).

I think the only viable option with Duplicati at the moment is to have multiple destinations for every backup set – that way when (I suspect ‘when’ is more correct than ‘if’ given the sheer amount of strange issues I’ve experienced) one of the destinations goes ‘bad’ it would be at least possible to copy files from another destination and continue to have working backups.

13

Sorry to hear it wasn’t recoverable, but I agree (also looking at you, CrashPlan) that the potential of Duplicati is it’s biggest benefit right now. :slight_smile:

I personally use two destinations, but that’s mostly because I keep abusing one of them for tests when working on this forum.

It really does seem like when Duplicati works, it really works, and when it doesn’t, it really doesn’t - I wish we could figure out why.

Until then,how did you use your copy from the other destination? Literally copy it over then recreate the database for the ‘broken’ job?

14

Yes, I deleted all the files from the broken job, copied all the files from another job, and rebuilt the database.

Is there any other option?

15

Sorry to jump in with no immediate help (those in this, please continue…) but I recently had the “Extra:” issue appear unexpectedly, so I took a technical look. Ultimately we want to understand the failure, so it can fixed…

duplicati-i03a0e9cd70ff44a292466283addb2b22.dindex.zip.aes: 20 errors
	Extra: +8vpCm4Muw2YV5qkqz6Phq37RmLDcioQjTVaEG0wtHQ=
	Extra: 0b8iZP2xvQ3bWfRsOkNVOGv+r9a7KChMFNlDfK0ixJw=
	Extra: 0mof4C7b71v6bcuQg8zDxWMeJ+Pq7JO21Knbzl7H824=
	Extra: 9CtMUlrfOMwd1vzyWXiK5yuwu6r5K2EN/YhXk2xy6e8=
	Extra: A91R/FY1WpWURm9TjA5SmPQBfn+LzG4/jBmrLbfqN4o=
	Extra: BfjzAmcEuBX7jYZJ+UoXeVpUgt+AScdWMx2Gn+1NoOc=
	Extra: FJzbL3YvV2CFDn0FT6ITrfaVI0Z4lNFHEArKgMj0h/w=
	Extra: QfeXtvB/L7IfvyKYvECeZj6BnEeBQ9hpTAXE2yIx9vo=
	Extra: bgG+tGJM2L3uysG01EBNAAuLYCgRKoZDWG+1wkW3sG4=
	Extra: e63i+inoVbTnXeaazsX15BLIXMJ+SMzvq2oCNNraIZs=
	... and 10 more


duplicati-b574cf4c8085e4c519d66c1e7c4974a66.dblock.zip.aes: 20 errors
	Extra: +8vpCm4Muw2YV5qkqz6Phq37RmLDcioQjTVaEG0wtHQ=
	Extra: 0b8iZP2xvQ3bWfRsOkNVOGv+r9a7KChMFNlDfK0ixJw=
	Extra: 0mof4C7b71v6bcuQg8zDxWMeJ+Pq7JO21Knbzl7H824=
	Extra: 9CtMUlrfOMwd1vzyWXiK5yuwu6r5K2EN/YhXk2xy6e8=
	Extra: A91R/FY1WpWURm9TjA5SmPQBfn+LzG4/jBmrLbfqN4o=
	Extra: BfjzAmcEuBX7jYZJ+UoXeVpUgt+AScdWMx2Gn+1NoOc=
	Extra: FJzbL3YvV2CFDn0FT6ITrfaVI0Z4lNFHEArKgMj0h/w=
	Extra: QfeXtvB/L7IfvyKYvECeZj6BnEeBQ9hpTAXE2yIx9vo=
	Extra: bgG+tGJM2L3uysG01EBNAAuLYCgRKoZDWG+1wkW3sG4=
	Extra: e63i+inoVbTnXeaazsX15BLIXMJ+SMzvq2oCNNraIZs=
	... and 10 more

What happened to me was that a compact put these blocks in a new dblock file, deleted some empty dblock files, then put these blocks into the next dblock file too. The database records them as belonging in the new file, so possibly I’m in the situation of it-works-but-it-complains, however I haven’t tried touching anything yet.

I downloaded dblock files, unencrypted, unzipped, and compared with WinMerge which identified all 20 dups. Unfortunately the deleted empty old dblock file wasn’t available for a certain look at what that had contained. Because the new dblock can only hold so much, at some time old dblock files can’t be fully emptied, so it will have to wait for the next dblock. I wasn’t sure how best to test a theory that the extras were a sequential run, however I tried sorting by sequential-looking ID in the Block table to see if extras were sequential. No, but in doing the lookup I saw they also weren’t Base64 of the SHA-256 hash I sometimes handle with tools like this. Instead, they were a character-mapped form, with a likely-troublesome forward slash becoming underscore, and plus becoming minus. The other characters in the Base64 table are letters and numbers and are safer.

What I find interesting is that all of my problems had this mapping, and those from the original post do too… Sampling my current block hashes, 49% have / and 49% have +, so it’s not rare, but to have 30 all show up this way, and 0 without the character mapping seems unlikely to be chance. Anyway, here are my 20 extras:

ID     Hash
355784 maGFet8d2IgpJQHiITVnsq-Ut-rA985ftbOl0lNfLao=
DB has maGFet8d2IgpJQHiITVnsq+Ut+rA985ftbOl0lNfLao=

355785 xVctf9vOEHW89Bk3xOzfiLADtczNMubXy-2LEF0AkDY=
DB has xVctf9vOEHW89Bk3xOzfiLADtczNMubXy+2LEF0AkDY=

355788 9CtMUlrfOMwd1vzyWXiK5yuwu6r5K2EN_YhXk2xy6e8=
DB has 9CtMUlrfOMwd1vzyWXiK5yuwu6r5K2EN/YhXk2xy6e8=

355789 e63i-inoVbTnXeaazsX15BLIXMJ-SMzvq2oCNNraIZs=
DB has e63i+inoVbTnXeaazsX15BLIXMJ+SMzvq2oCNNraIZs=

355790 vg3NXOItS6baRcHUCVPQrRyb69zKLXsbUvQLgv_ViBA=
DB has vg3NXOItS6baRcHUCVPQrRyb69zKLXsbUvQLgv/ViBA=

355791 FJzbL3YvV2CFDn0FT6ITrfaVI0Z4lNFHEArKgMj0h_w=
DB has FJzbL3YvV2CFDn0FT6ITrfaVI0Z4lNFHEArKgMj0h/w=

355792 goT-apmyALynpED2C2n0vUQ5Ua6TKk8cFuU5iJ5Ph28=
DB has goT+apmyALynpED2C2n0vUQ5Ua6TKk8cFuU5iJ5Ph28=

355793 0mof4C7b71v6bcuQg8zDxWMeJ-Pq7JO21Knbzl7H824=
DB has 0mof4C7b71v6bcuQg8zDxWMeJ+Pq7JO21Knbzl7H824=

355796 -8vpCm4Muw2YV5qkqz6Phq37RmLDcioQjTVaEG0wtHQ=
DB has +8vpCm4Muw2YV5qkqz6Phq37RmLDcioQjTVaEG0wtHQ=

355797 x2n8TLucw2rPJxEHwaBKS2SVS13z-hFmf4fM6oFGTUk=
DB has x2n8TLucw2rPJxEHwaBKS2SVS13z+hFmf4fM6oFGTUk=

355802 sGxIQhFRvtfH9ZaHv-tocxtXRyZe17tEiZVhRrPou3g=
DB has sGxIQhFRvtfH9ZaHv+tocxtXRyZe17tEiZVhRrPou3g=

355803 0b8iZP2xvQ3bWfRsOkNVOGv-r9a7KChMFNlDfK0ixJw=
DB has 0b8iZP2xvQ3bWfRsOkNVOGv+r9a7KChMFNlDfK0ixJw=

355804 BfjzAmcEuBX7jYZJ-UoXeVpUgt-AScdWMx2Gn-1NoOc=
DB has BfjzAmcEuBX7jYZJ+UoXeVpUgt+AScdWMx2Gn+1NoOc=

355805 bgG-tGJM2L3uysG01EBNAAuLYCgRKoZDWG-1wkW3sG4=
DB has bgG+tGJM2L3uysG01EBNAAuLYCgRKoZDWG+1wkW3sG4=

355817 A91R_FY1WpWURm9TjA5SmPQBfn-LzG4_jBmrLbfqN4o=
DB has A91R/FY1WpWURm9TjA5SmPQBfn+LzG4/jBmrLbfqN4o=

355818 f31-zQkFF37LIzGqG0H7aN9aj7M5h4UiGl0_OI-eDhs=
DB has f31+zQkFF37LIzGqG0H7aN9aj7M5h4UiGl0/OI+eDhs=

355819 QfeXtvB_L7IfvyKYvECeZj6BnEeBQ9hpTAXE2yIx9vo=
DB has QfeXtvB/L7IfvyKYvECeZj6BnEeBQ9hpTAXE2yIx9vo=

355820 myL8bELIN48gNgWmv31-vhjd8t4JyiXD2xXDX7FXFag=
DB has myL8bELIN48gNgWmv31+vhjd8t4JyiXD2xXDX7FXFag=

355821 i9eUzDmllQC9icHoFh5kU-OSOL2kvGrlQOJAJxQeuwc=
DB has i9eUzDmllQC9icHoFh5kU+OSOL2kvGrlQOJAJxQeuwc=

355823 ipvzQcfL4oolUhq-v_1TekFrFCpQzOGOU3ERZkkDUpE=
DB has ipvzQcfL4oolUhq+v/1TekFrFCpQzOGOU3ERZkkDUpE=

and if the guess about character mapping isn’t helpful, I have another guess to offer :wink:, which is that the dblock file that was disassembled for compaction contains an empty file (dealing a lot with those recently), which is apparently done wrong as 0 bytes packed size, according to 7-Zip whose test calls it a Data error. Windows File Explorer also complained about it, but I had it skip that file, and it kept on extracting the rest.

Here’s the log with an asterisk before the two mentioned files. This was another case of a hidden problem being inadvertently inserted into what seemed like a successful backup, until later work showed otherwise.

2018-12-17 20:32:47 -05 - [Information-Duplicati.Library.Main.Database.LocalDeleteDatabase-CompactReason]: Compacting because there is 27.32% wasted space and the limit is 25%
2018-12-17 20:32:48 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-be7bc2b30f65a4cfa9500766aac617df1.dblock.zip.aes (16.27 MB)
2018-12-17 20:32:53 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-be7bc2b30f65a4cfa9500766aac617df1.dblock.zip.aes (16.27 MB)
2018-12-17 20:32:53 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b50f53e3980194661b927f067b4c96760.dblock.zip.aes (50.07 MB)
2018-12-17 20:33:07 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b50f53e3980194661b927f067b4c96760.dblock.zip.aes (50.07 MB)
2018-12-17 20:33:07 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b487235807edf44a1ab74ac1a8b102239.dblock.zip.aes (50.06 MB)
2018-12-17 20:33:19 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b487235807edf44a1ab74ac1a8b102239.dblock.zip.aes (50.06 MB)
2018-12-17 20:33:19 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b67b63d7968e349bfa7f24b3058c2a8f2.dblock.zip.aes (14.93 MB)
2018-12-17 20:33:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b67b63d7968e349bfa7f24b3058c2a8f2.dblock.zip.aes (14.93 MB)
2018-12-17 20:33:22 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-bc84a3eeda0cc45f48678c5c97b574f4f.dblock.zip.aes (50.02 MB)
2018-12-17 20:33:38 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-bc84a3eeda0cc45f48678c5c97b574f4f.dblock.zip.aes (50.02 MB)
2018-12-17 20:33:38 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b8a294b40162e4053998d1a3251809956.dblock.zip.aes (3.04 MB)
2018-12-17 20:33:40 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b8a294b40162e4053998d1a3251809956.dblock.zip.aes (3.04 MB)
2018-12-17 20:33:43 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b574cf4c8085e4c519d66c1e7c4974a66.dblock.zip.aes (50.04 MB)
*2018-12-17 20:35:25 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b574cf4c8085e4c519d66c1e7c4974a66.dblock.zip.aes (50.04 MB)
2018-12-17 20:35:25 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i03a0e9cd70ff44a292466283addb2b22.dindex.zip.aes (49.18 KB)
2018-12-17 20:35:26 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i03a0e9cd70ff44a292466283addb2b22.dindex.zip.aes (49.18 KB)
2018-12-17 20:35:27 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-be7bc2b30f65a4cfa9500766aac617df1.dblock.zip.aes (16.27 MB)
2018-12-17 20:35:27 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-be7bc2b30f65a4cfa9500766aac617df1.dblock.zip.aes (16.27 MB)
2018-12-17 20:35:27 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ia3feabd6d45043b889fdaea2426c8312.dindex.zip.aes (13.89 KB)
2018-12-17 20:35:27 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ia3feabd6d45043b889fdaea2426c8312.dindex.zip.aes (13.89 KB)
2018-12-17 20:35:27 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-b50f53e3980194661b927f067b4c96760.dblock.zip.aes (50.07 MB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-b50f53e3980194661b927f067b4c96760.dblock.zip.aes (50.07 MB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-ifdf5b6929b2749678ee50fa14450a34e.dindex.zip.aes (27.58 KB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-ifdf5b6929b2749678ee50fa14450a34e.dindex.zip.aes (27.58 KB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-b487235807edf44a1ab74ac1a8b102239.dblock.zip.aes (50.06 MB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-b487235807edf44a1ab74ac1a8b102239.dblock.zip.aes (50.06 MB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-icd2bbc0418044537a63989002fdc3a13.dindex.zip.aes (34.43 KB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-icd2bbc0418044537a63989002fdc3a13.dindex.zip.aes (34.43 KB)
2018-12-17 20:35:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-b67b63d7968e349bfa7f24b3058c2a8f2.dblock.zip.aes (14.93 MB)
2018-12-17 20:35:29 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-b67b63d7968e349bfa7f24b3058c2a8f2.dblock.zip.aes (14.93 MB)
2018-12-17 20:35:29 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-i852cb8aaf2c947c1a4d8abccafac3b85.dindex.zip.aes (8.84 KB)
2018-12-17 20:35:29 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Completed: duplicati-i852cb8aaf2c947c1a4d8abccafac3b85.dindex.zip.aes (8.84 KB)
2018-12-17 20:35:29 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b9296f0666088421e888e8ddc50db95a9.dblock.zip.aes (2.47 MB)
2018-12-17 20:35:32 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b9296f0666088421e888e8ddc50db95a9.dblock.zip.aes (2.47 MB)
2018-12-17 20:35:32 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-be09724cfc36f4301b332ac92f6391f4f.dblock.zip.aes (5.60 MB)
2018-12-17 20:35:34 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-be09724cfc36f4301b332ac92f6391f4f.dblock.zip.aes (5.60 MB)
2018-12-17 20:35:34 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b62f47a520ab34843b52d0a168a523dab.dblock.zip.aes (3.54 MB)
2018-12-17 20:35:36 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b62f47a520ab34843b52d0a168a523dab.dblock.zip.aes (3.54 MB)
2018-12-17 20:35:36 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b4cf18ef7c8464d1c89b761b2067386a9.dblock.zip.aes (3.16 MB)
2018-12-17 20:35:37 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b4cf18ef7c8464d1c89b761b2067386a9.dblock.zip.aes (3.16 MB)
2018-12-17 20:35:37 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-be4f42d062303483095dbf298737d00b5.dblock.zip.aes (4.29 MB)
2018-12-17 20:35:38 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-be4f42d062303483095dbf298737d00b5.dblock.zip.aes (4.29 MB)
2018-12-17 20:35:38 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-be82052fbf7384bef95c807ad171180f4.dblock.zip.aes (7.28 MB)
2018-12-17 20:35:40 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-be82052fbf7384bef95c807ad171180f4.dblock.zip.aes (7.28 MB)
*2018-12-17 20:35:42 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bf8ac8783fc394b22a22ca3eb888c68d9.dblock.zip.aes (34.35 MB)

Because this is sort of a production backup, I just exported it into a new job, so the old can be studied more.

EDIT: The theory about / and + translation is somewhat supported and somewhat refuted by other reports…

Question about TEST command vs. Backup with –full-remote-verification and backup-test-samples
10 extras, 0 without / or +

Verify a specific dblock.zip.aes file
8 extras, 4 without / or +

SharpCompress.Compressors.LZMA.DataErrorException
20 extras, 0 without / or +

“Unexpected number of remote volumes marked as deleted” error
8 extras, 2 without / or +, not counting a bizarre one with a dblock file name where the hash usually is.

16

Sorry to relive that old topic

I agree with this. This is what I end doing with two issues.

I do the same but to avoid the DB rebuilt time, I copy the local DB of the working backup, change the name of the DB file (XXXX.sqlite). After configuring the new backup (I do not export import settings as it has caused me some issues), I change the DB path to the copied DB. It works directly.