During a moment of total stupidity, I accidentally removed a few hundred remote files in my b2 bucket.
Quite logically, Duplicati has complained about that during the next backup command, suggesting to repair the backup.
I tried the --rebuild-missing-dblock-files option but this one complained about corrupted dblocks, and suggested to list broken files and then purge them.
This is what I did next, and while it worked without any error output, the backup does not want to work either.
I get this kind of error messages:
Fatal error => Found inconsistency in the following files while validating database:
/mnt/backup/share/obones/doc and settings/obones/Application Data/XnView/XnView.db, actual size 164340736, dbsize 150516736, blocksetid: 497524
/mnt/backup/share/obones/doc and settings/obones/Application Data/XnView/original1.XnView.db, actual size 164340736, dbsize 150516736, blocksetid: 497524
/mnt/backup/share/obones/doc and settings/obones/Application Data/Winamp/Winamp.m3u8, actual size 78, dbsize 0, blocksetid: 497531
/mnt/backup/share/obones/doc and settings/obones/Application Data/Winamp/Winamp.q1, actual size 4587, dbsize 0, blocksetid: 497533
/mnt/backup/share/obones/doc and settings/obones/Application Data/Winamp/links.xml, actual size 3259, dbsize 0, blocksetid: 497535
... and 83445 more. Run repair to fix it.
So, hey, I then went on to run repair and this one goes through apparently fine, without any output after the Listing remote files line.
But this did not seem to help at all because I’m getting the exact same error message when I start backup again.
What should I try next?
I mean, I can always empty the entire remote bucket, delete the database file and start over. But that would mean losing the backup history which I’d like to avoid if possible.
If it’s only 1 day, it’s already too late, but for the life of me, I can’t even find how to unhide a file from B2’s website.
I mean, I know that a file is “hidden” by a special version for it, but where do I find those files is a total mystery to me.
You can see the hidden (soft deleted) files in the B2 Web UI. Undeleting one at a time would be painful. Maybe there’s a command line utility that would help. Sorry to hear it’s too late.
Was that in the B2 web UI or with a tool? Some tools, e.g. rclone are more savvy about versions/hiding.
You could ask Backblaze support if their web GUI can show these. I think their Command-Line Tool can. Cyberduck can also show hidden files, and probably others can too (but Duplicati does delete, not hide).
If you did a Backblaze web GUI delete, I’d guess that’s a delete, but you could confirm it with Backblaze.
Do you know if there was any pattern to your delete? That might give a clue as to how bad damage was.
Deleted dindex and dlist files can sometimes be uploaded if there is a Duplicati database with their data.
Deleted dblock files generally mean source file blocks are gone. Usually one purges those “broken” files, however you ran into other issues nearby, somehow getting some other inconsistencies in the database.
Duplicati.CommandLine.RecoveryTool.exe can deal somewhat better with damaged backups, however missing backup data remains missing, and I “think” the result is that some restored files are incomplete.
Unattractive options are to get what history you can right now, or save the damaged backup for future try.
It may or may not be possible to turn the remaining backup files into a database, but you can copy off the current Database and see how badly using the Recreate button goes. It will almost surely dislike missing files, and might download all files in the backup looking for missing blocks. If you have some drive space, downloading everything to local drive would avoid repeated B2 download charges.if this takes a few tries.
Getting a good view of the likely Recreate failure might help decide future steps. I’m not sure how well the normal logs capture them. It would probably be worth setting up log-file=<path> log-file-log-level=Verbose before trying the Recreate. You can also watch action on About → Show log → Live → Verbose. Progress bar on screen will be in the 90%-100% range and moving slowly as it searches through all the dblock files. Search completion time can be roughly estimated from the numbers on the verbose log messages there.
This was with a tool, you know, the Duplicati.CommandLine.exe one. Basically, I wanted to repair another backup but I did not use the proper destination bucket for the database file that I gave. And before I figured that out, it already had deleted 700 files in the bucket.
Well, I did just that, copied the 3GB database to another name and used the Recreate option in the web GUI. It failed quite fast saying No filelists found on the remote destination
So basically, I believe I’m left with the only possible situation where I delete all files in the B2 bucket and restart from scratch.
Well, that’ll teach me to double check my command line arguments in the future
