Unable to restore any files after harddrive crash

After this I will stop using Duplicati.

I’m getting a similar issue when I try to do a restore from my google drive backup set via “direct restore from backup files”. The database rebuilds (i’m on the latest canary and there’s only ~4GB there so it does this quickly), then reports one backup file missing in my case, throws an error, gives a “restore complete” screen but hasn’t actually restored anything.

For comparison, I restored straight from the saved backup set - and chose the same, small, top-level file, and it worked without issue.

Paging other mods (@JonMikelV, @kees-z, @tophee) in case anyone has any insight and/or suggestions for @mdwyer’s case.

I made a backup of the rebuilt database and I am rebuilding it AGAIN (6 times now) from scratch.
I tried various repair options but nothing seems to fix the missing files message.

The main files I want to restore are tiny but there seems to be no easy way to get them back. I’m wishing Duplicati didn’t need to rebuild the entire thing each and every time. Yes I have a database now after tearing my hair out over the temporary useless database, but my files may as well be on the Moon at this point.

Here is a crazy idea for a feature request… Have Duplicati upload its database to the backup folder after completing a backup. If it did that, I wouldn’t need to waste days rebuilding it.

1 Like

Despite my earlier comment about not using Duplicati after this, I actually like it. When it works it is great. I use it at work and it has saved the day twice so far with ransomware striking some laptops. However when it doesn’t work, it is an absolute nightmare to fix not helped by the high stress of needing those backups restored asap.

I badly need a tool that allows me to navigate the backups, flag a file and extract it without needing the entire database rebuilt and in perfect health. I am quite certain that there is nothing wrong with the backup where my files are but faults elsewhere in it are blocking my attempts to retrieve them.

3 Likes

I’m not sure I remember where, but I’m pretty sure I’ve seen others around here mention a restore tool which can be used to restore files without needing to do a full DB repair, or something like that. I haven’t used it myself though - you might search around old posts for info on this if you haven’t already.

I think you mean the RecoveryTool.exe which is part of the installation (including built-in documentation):

While it does work without the database, it is really slow due to it not having a database. It is meant for disaster recovery, not for a simple “lets extract a single file”.

The problem with the “extract one file” is that the information is scattered. The dlist files contains the filenames and hashes, so you need to start there. Once you have the hashes, you need to find the dblock files that contain these blocks.

Without a database, you pretty much have to look into (aka download) each dblock file and see if it has what you want.

1 Like

Thanks, that’s probably what I was thinking of. It seems like @mdwyer might need this tool, or at least futher help with this restore issue.

If your main objective is to restore from Duplicati then move on to something else, your best bet is probably to use the RecoveryTool.exe suggested by kenkendk.

If you’re thinking you might still want to use Duplicati, then we need to figure out what’s causing the “missing files” error and resolve that.

For example, if the files really are missing then they should be purged from the backups/ However, if you manually check the destination and verify (at least some of) the specific files are really there, then we need to figure out why Duplicati isn’t able to see them.

When you rebuilt the database and then got the missing files error again, do you recall if it was still 85 files or if it seemed like the same files?

To what destination are you backing up and is it possible your files are automatically being archived to cold storage or something like that? Scratch that last question, I see you’ve already said you’re backing up to your local G: drive.

Would it be possible for you to post one or two of the actual error messages?

I already tried the recovery tool. I used it several times and many of its options. I googled all of this before posting and tried everything I found. It also failed to recover any files.

Here is the console output from some of my attempts to rebuild/restore. Other similar attempts using the other recovery tool were done on a different computer.

Console log 1

Console log 2

Here are the various commands I tried… console commands

I am currently re-running a database repair via the web interface. I told it to delete the database and rebuild. It will be complete in a day or so.

FWIW, recent builds allegedly do DB repair much more quickly - if you feel like fucking with your installed versions (and i’d understand if you’re hesitant), you might try that and see if it expedites your attempts any.

In one of my logs it complains about versions mentioning version 7 and 4. I gather my backup is version 7 but I’m not sure about that or which version I should attempt to use. I did try installing via the web interface some of the dev builds but it only downloads them. It did not seem to make any attempt to install them.

I have at least one completed database rebuild backed up on my G drive. The current live database is a completely new rebuild which may finish by midnight tonight. My next move will be to make a copy of that before proceeding.

The auto-updater has been fixed quite a bit in the more recent canaries. If you’re willing to hassle with it, I’d recommend downloading the installer for the most recent one directly and running that. I don’t happen to know of any issues the newer versions would cause when trying to access your saved backup set (and the last canaries are fairly stable in my experience).

I’ll try that shortly. The latest rebuild has finished, I have backed up the database and I’m attempting to restore a file. After that I’ll try updating Duplicati.

The restore failed again. At the end of the db rebuild it gives this error…

Could not find file ‘G:\beast_backups\aes\duplicati-bf11547ab15ad4008aa3a543ec54550f8.dblock.zip.aes’.

The actual restore process sat for minutes eventually doing a verifying remote files stage after which it errors with the message about the missing 85 files and nothing is restored.

Why doesn’t the db rebuild deal with all of that? It seems that rebuilding the database is simply step 1 of many steps.

I installed the latest Canary and…

C:\Windows\System32>"C:\Program Files\Duplicati 2\Duplicati.Server.exe"
A serious error occurred in Duplicati: System.IO.FileNotFoundException: Could not load file or assembly ‘CoCoL, Version=1.5.0.24305, Culture=neutral, PublicKeyToken=0983de3c914beeaa’ or one of its dependencies. The system cannot find the file specified.
File name: 'CoCoL, Version=1.5.0.24305, Culture=neutral, PublicKeyToken=0983de3c914beeaa’
at Duplicati.Library.UsageReporter.Reporter.Initialize()
at Duplicati.Library.UsageReporter.Reporter.SetReportLevel(Nullable`1 maxreportlevel, Boolean disable)
at Duplicati.Server.Program.StartOrStopUsageReporter()
at Duplicati.Server.Program.RealMain(String[] args)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].

Crash!
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. —> System.IO.FileNotFoundException: Could not load file or assembly ‘CoCoL, Version=1.5.0.24305, Culture=neutral, PublicKeyToken=0983de3c914beeaa’ or one of its dependencies. The system cannot find the file specified.
at Duplicati.Library.UsageReporter.Reporter.ShutDown()
at Duplicati.Server.Program.RealMain(String[] args)
— End of inner exception stack trace —
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Duplicati.Library.AutoUpdater.UpdaterManager.RunMethod(MethodInfo method, String[] args)

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly ‘CoCoL, Version=1.5.0.24305, Culture=neutral, PublicKeyToken=0983de3c914beeaa’ or one of its dependencies. The system cannot find the file specified.
at Duplicati.Library.AutoUpdater.UpdaterManager.RunMethod(MethodInfo method, String[] args)
at Duplicati.Library.AutoUpdater.UpdaterManager.<>c__DisplayClass58_0.b__0()
at Duplicati.Library.AutoUpdater.UpdaterManager.WrapWithUpdater(AutoUpdateStrategy defaultstrategy, Action wrappedFunction)
at Duplicati.Library.AutoUpdater.UpdaterManager.RunFromMostRecentSpawn(MethodInfo method, String[] cmdargs, AutoUpdateStrategy defaultstrategy)
at Duplicati.Server.Program.Main(String[] args)

I got this version to run without crashing… v2.0.2.15-2.0.2.15_experimental_2018-01-03

How do I fix the missing files problem?

1 Like