I have all of my backup files spanning several months and I have repaired the database but every time I attempt to restore even 1 tiny file, it says it couldn’t find 85 backup files and to run repair (I have). It still says that.
It then says it has restored the files but no files are restored. I tell it to restore to a specific folder which exists, not the original locations. I can’t restore to original locations, it is a new harddrive with everything on it new.
I am at a loss what to do and don’t really understand how any files can be missing. I’m more concerned about Duplicati’s complete inability to restore anything than why it reports some missing files. I have a lot of backup files. Why can’t they produce anything?
This sounds like an issue for @kenkendk to look at, but before he gets here I would suggest you provide some additional info to help troubleshoot:
What backup destination were you using and/or where are the files currently located (if different)?
What exact steps did you do when trying your unsuccessful database repair?
I would advise you that in my previous tests with (in my case) unreliable restore sources, a failed restore still gave a “restore successful” message, which I think is just incorrect wording within Duplicati, so when duplicati says it has restored your files, take it with a grain of salt.
I have tried to restore different single files a few times. It always complains about missing 85 files and does nothing. Is there any way (that works) to get past that message and have it actually restore files? I have a lot more than 85 backup files, over 2,000 Duplicati backup files. It feels like 85 mysteriously missing files are holding my backup to randsom with no way forward.
I have the backups on my local G drive and I am restoring to a folder called rescue on the same drive.
I am running 2.0.2.1_beta_2017-08-01 on Windows 10 and using the web interface. I previously rebuilt the database using the command line since the web interface only ever made a temporary database which it immediately lost with no chance to recover it. It takes 2 full days to rebuild the database. I have been rebuilding it multiple times since my drive crash on Tuesday last week.
I select the backup, click restore, wait, select 1 file which is in the root (not buried in a folder) in the list of backed up files, select the destination and click restore. It doesn’t take long before I get the message about the missing files and the ‘successful’ restore message.
I’m getting a similar issue when I try to do a restore from my google drive backup set via “direct restore from backup files”. The database rebuilds (i’m on the latest canary and there’s only ~4GB there so it does this quickly), then reports one backup file missing in my case, throws an error, gives a “restore complete” screen but hasn’t actually restored anything.
For comparison, I restored straight from the saved backup set - and chose the same, small, top-level file, and it worked without issue.
I made a backup of the rebuilt database and I am rebuilding it AGAIN (6 times now) from scratch.
I tried various repair options but nothing seems to fix the missing files message.
The main files I want to restore are tiny but there seems to be no easy way to get them back. I’m wishing Duplicati didn’t need to rebuild the entire thing each and every time. Yes I have a database now after tearing my hair out over the temporary useless database, but my files may as well be on the Moon at this point.
Here is a crazy idea for a feature request… Have Duplicati upload its database to the backup folder after completing a backup. If it did that, I wouldn’t need to waste days rebuilding it.
Despite my earlier comment about not using Duplicati after this, I actually like it. When it works it is great. I use it at work and it has saved the day twice so far with ransomware striking some laptops. However when it doesn’t work, it is an absolute nightmare to fix not helped by the high stress of needing those backups restored asap.
I badly need a tool that allows me to navigate the backups, flag a file and extract it without needing the entire database rebuilt and in perfect health. I am quite certain that there is nothing wrong with the backup where my files are but faults elsewhere in it are blocking my attempts to retrieve them.
I’m not sure I remember where, but I’m pretty sure I’ve seen others around here mention a restore tool which can be used to restore files without needing to do a full DB repair, or something like that. I haven’t used it myself though - you might search around old posts for info on this if you haven’t already.
I think you mean the RecoveryTool.exe which is part of the installation (including built-in documentation):
While it does work without the database, it is really slow due to it not having a database. It is meant for disaster recovery, not for a simple “lets extract a single file”.
The problem with the “extract one file” is that the information is scattered. The dlist files contains the filenames and hashes, so you need to start there. Once you have the hashes, you need to find the dblock files that contain these blocks.
Without a database, you pretty much have to look into (aka download) each dblock file and see if it has what you want.
If your main objective is to restore from Duplicati then move on to something else, your best bet is probably to use the RecoveryTool.exe suggested by kenkendk.
If you’re thinking you might still want to use Duplicati, then we need to figure out what’s causing the “missing files” error and resolve that.
For example, if the files really are missing then they should be purged from the backups/ However, if you manually check the destination and verify (at least some of) the specific files are really there, then we need to figure out why Duplicati isn’t able to see them.
When you rebuilt the database and then got the missing files error again, do you recall if it was still 85 files or if it seemed like the same files?
To what destination are you backing up and is it possible your files are automatically being archived to cold storage or something like that? Scratch that last question, I see you’ve already said you’re backing up to your local G: drive.
Would it be possible for you to post one or two of the actual error messages?
I already tried the recovery tool. I used it several times and many of its options. I googled all of this before posting and tried everything I found. It also failed to recover any files.
Here is the console output from some of my attempts to rebuild/restore. Other similar attempts using the other recovery tool were done on a different computer.
FWIW, recent builds allegedly do DB repair much more quickly - if you feel like fucking with your installed versions (and i’d understand if you’re hesitant), you might try that and see if it expedites your attempts any.
In one of my logs it complains about versions mentioning version 7 and 4. I gather my backup is version 7 but I’m not sure about that or which version I should attempt to use. I did try installing via the web interface some of the dev builds but it only downloads them. It did not seem to make any attempt to install them.
I have at least one completed database rebuild backed up on my G drive. The current live database is a completely new rebuild which may finish by midnight tonight. My next move will be to make a copy of that before proceeding.
The auto-updater has been fixed quite a bit in the more recent canaries. If you’re willing to hassle with it, I’d recommend downloading the installer for the most recent one directly and running that. I don’t happen to know of any issues the newer versions would cause when trying to access your saved backup set (and the last canaries are fairly stable in my experience).