Restore files from Duplicati

We are having trouble Restoring our files from duplicati after a major server outage.
The error we are getting is “Failed to decrypt data (Invalid passphrase?): invalid password or Corrupted data”
The passphase we are using is copied and pasted, and if we put an invalid passphrase in it doesn’t come up with a date of last backup. So we are cconfident we dont have an issue there.
I have run the command line tool to repair the data and look for broken files but these appear to be sitting there doing nothing
What are your suggestions please

Regards
Kevin

Bump!
Could our problem be caused because there is no database
and not able to rebuild the database

Regards,
Kevin

I read too many posts about problems restoring, and it worries me. All those database problems. Is Duplicati robust enough as a backup system?

I’m not saying it’s not and after all I’m using it, albeit in parallel for now, and the odd file restore has worked well, but I’ve not had to rely on it yet IYSWIM.

You should be able to restore directly from the destination without having to manually recreate the database however this uses a temporary database (presumably made for this restore). You eventually want a full one, however which way to go depends on the situation, and it wasn’t clear what was lost, reinstalled, or restored.

Restoring files from a backup is just an ordinary restore. Do you have any idea when Restore used to work?

Restoring files if your Duplicati installation is lost covers the case of severe loss, e.g. lost the source system.

Disaster Recovery is more aimed at problems within the backup than problems a sound backup means to fix.

What OS and Duplicati versions is this? Are destination files accessible? What suffix do their names end in?

If you are doing any operations from a shell or Command Prompt, be careful about special characters which might be part of the passphrase. Using the web UI (or the Commandline option in UI) should avoid the issue, and a quoted version can be obtained from the web UI using Configuration → Export → As Command-line. Rather than quote, you can use the –parameters-file option instead of placing everything on command line.

It’s important to match relevant backup settings of the UI, especially if running from a true command line not the one inside the UI which helpfully populates the settings from the backup job, then leaves it to you to get the operation adjusted exactly to the command syntax per “help” or Using Duplicati from the Command Line.

Your test for invalid passphrase adds some confidence however the date is also right on the file name, so it isn’t solid proof (without going through the code…). Better proof would be if you can select that backup and browse the files inside. The file list comes from the dlist file of the right date, and is encrypted if you encrypt.

You can also verify your passphrase using command line SharpAESCrypt.exe from the Duplicati installation, or https://www.aescrypt.com/ run on a copy of some file (maybe that dlist?) copied from the destination area.

It’s not clear exactly when the error happened. Various things are decrypted, and it would help to know what failed. A clearer view than the basic web UI provides can be obtained in the UI using About → Show log → Live in another tab (set to level Information or even level Profiling, but Profiling could make a LOT of output). Running on the command line also makes a nicer display, and can be done with web UI Commandline option after setting some options to get more output. It’s either –log-level or –console-log-level if on recent canary.

The enhanced logging settings can also be used on true command line, however the operation is separate from the web UI, and there are hazards to running independent copies of Duplicati to the same destination. For the already-launched seemed-stuck true-command line (correct?) tool, it might be best to look at it with other tools to see where it is. Depending on OS, this might include Task Manager, Process Explorer, ps, etc. Basically, stopping it in the middle of work might be bad, but if it’s truly stalled then it’s probably appropriate. You can also look for database progress, e.g. timestamps. Location depends on your OS and how Duplicati was installed, e.g. as Windows service or Linux service to start at boot, or using Tray Icon and starting later.

Hearing more about the situation and sequence of things done would also help, for example if you truly lost everything on the drive, then a “repair” would try to recreate the database, which can take a long time for a big backup, and as noted earlier isn’t immediately required if you’re in a hurry to restore files from a backup.

Answers to Questions

it wasn’t clear what was lost, reinstalled, or restored

Due to issues at the data centre we are currently unable to gain access to the servers

Do you have any idea when Restore used to work?

A Restore was done a week before the data center went down

What OS and Duplicati versions is this?

Duplicati was and still is on Server 2012 R2 and Duplicati - 2.0.3.11_canary_2018-09-05

Are destination files accessible?

They are on my local Laptop and are accessed via ftp (I checked the ftp connection this morning). We have also copied direct onto the server to avoid any connectivity issues

What suffix do their names end in?

*.zip.AES

Passphrase

We are 100% satisfied this is correct

What’s the relationship of the data centre to the server in question? I’m just looking to understand history.

Was it like this? Note I’m also trying to understand the destination being restored from, normally and now, because every time a transfer is done, especially over a somewhat unreliable protocol like FTP, it’s a risk.

Possible summary:

Server 2012 R2 with Duplicati does normal backup to a laptop over FTP. Restore from it worked recently.

Server drive totally lost. Duplicati was reinstalled in order to restore files, however without any databases.

A backup job export was not available to import back in, so selected settings were pasted in from records.

In addition to direct connect with laptop, some binary-safe transfer (FTP is not safe when in ASCII mode) copied all of the laptop destination’s files onto the server, so they could be directly accessed by Duplicati.

Is above summary right? If so, I don’t think I’m retracting any prior suggestions (which might have to wait awhile if there’s a problem getting to the server now), but instead of decrypting to check the passphrase, decrypting would be done to check file integrity. Or maybe I can think of an easier way to check all files…

EDIT: I’m still thinking decryption would be a good first check on whether there are actually any bad files. Scripted SharpAESCrypt is one option, but I discovered AES Crypt can handle File Explorer multi-select. Running on original destination files would add risks and mess, so if you do this, please use copied files. Another option is Duplicati.RecoveryTool.exe per Recovering by using the Duplicati Recovery tool, which has a variety of capabilities including downloading from destination, decrypting, and restoring files.

You shouldn’t normally have to do this. Test main menu Restore → Direct restore from backup files first, however if that still fails we’d have to check for actual file corruption, try to see what got hit, find how, etc.