Lost my duplicati backup configuration. I could not figure out how to recover it

Support
1

Just some feedback for you.

Consider the usage scenario: user uses a NAS connected by USB to a wifi router, and uses a duplicati backup configuration file located on a windows laptop, and the backup works successfully for years.

Then, the hard disk crashes on the windows laptop HARD. It cannot be read in any way.

I installed a new SSD on the laptop, and fortuneately, I was able to restore the backup from the NAS.

However, I lost one file and that is the duplicati backup configuration file. I could not figure out how to get it. The UI for duplicati suggested that the backup configuration file was in the backup files themselves on the NAS. I tried that, but, it failed.

2

Please post words or picture. There should be no such suggestion made, because it would be false.
The local job database can generally be created from destination (see below), but you ask for config.
That’s the information you type in when you add a backup job. It’s not clear which of those you want.
Is the end goal to keep using the backup? That needs database plus at least the approximate config.

Exporting a backup job configuration and saving it securely is the way to save an exact configuration.
Without that, recovery can be approximated by looking at the restored information to see what it had.
Is there some particularly difficult aspect? Some people confuse the configuration with job database.

How? Did you use Restoring files if your Duplicati installation is lost? If so, you need the job database.
If, on the other hand, you typed a rough configuration then did a Repair, maybe you’re tweaking config.

3

Hi there :slight_smile:

I’m new to Duplicati and I also thought initially that I could restore a configuration from the remote backup, because of what’s written on the UI:

image
image844Ă—212 22.1 KB

The phrase “Load a configuration from an exported job or a storage provider” led me to think that I was going to be able to point to my remote backup, and that its configuration could have been restored. But, unless I’m missing something, the only option presented on the next screen seems to be restoring a configuration from an exported job:

image
image987Ă—339 28.6 KB

Is this still not possible in 2024? If not, wouldn’t that be a very handy feature? Reconfiguring everything from scratch manually can be quite cumbersome, especially in my case because I had dozens of configured filters as regular expressions, etc.

I feel like that would be nice if the remote backup also stored the backup configuration, making it possible to recreate a backup Job automatically from it :slight_smile:

Cheers.
PS: I’m running Duplicati - 2.1.0.2_beta_2024-11-29 on Ubuntu 24.04.1 LTS

4

This request comes up occasionally. There’s at least one official GitHub enhancement request:

Automatically Include Configuration in Backup #3073

Interestingly an undocumented option has been in the code for over 10 years now according to

Entire configuration backup (which has some caveats on it) and the 2014 pull request adding it.

I can see the 2017 change that added the text you cite, but I’m not sure how well it’s supported.

Maybe the developer will share thoughts. Meanwhile, is this for planning ahead or lost already?

As far as I know, there are ways to protect configs, but none I’d call default or well documented.

The manual just got rewritten, and it doesn’t have much to say:

Import and export backup configurations

While it is not required that you keep a copy of the backup configuration, it can sometimes be convenient to have all settings related to a backup stored in a single file.

Exporting a backup job configuration is the old manual. Both have some notes on sensitive data.

At least one person posted that they backup their Duplicati-server.sqlite, with all the configs. Some months ago, though, the intent was to tie these to systems. That would break this method.

The new ServerUtil program has “Export a backup configuration”. If you get an export, you could probably copy it somewhere safe, or even attach it to Duplicati destination using the documented
control-files to put it in the dlist file for each version.This is what the undocumented option does.

If I use the undocumented option:

ZIP file has an extra folder with task-setup.json file in it that looks to me like it’s that job export,
and I think that’s the same folder that control-files uses, but I haven’t run it lately to be certain.

1 Like
5

Thanks for your fast reply :slight_smile:

For planning ahead. In fact, I’ve lost the complete installation of Duplicati, its configurations and its databases. I’ve reinstalled everything, and reconfigured everything manually, as I could not figure out a way to recover configuration from the remote backup files.

So, everything’s good for now on my end, but I was wondering if I could restore Duplicati’s configurations way quicker and easier in the future.

Well, at least for regular users, the UI doesn’t seem to give the option although it’s mentioning it, so that’s quite confusing.

Thanks for all the links and references.

With regards to:

I get @kenkendk’s point, but in my opinion, it really depends on the context. In my case, I would have been glad to recover all my configurations, especially the large list of regexp filters and folders to include/exclude. My backup includes most of what’s in my /home/me directory, but some do have to be excluded like downloads, trash, cache, steam games, and so on… If I’m restoring my backup, even if it’s on a new Ubuntu installation, chances are I will want the same configuration, or at least something very similar to it. Having the option not to start from scratch would be great.

I’ll go through what the ServerUtil program can offer and see if I can automate something.

Thanks for your great answer @ts678.

6

The ServerUtil utility turned out to be pretty useful! :+1:

So here is what I did in case anyone’s interested:

I configured my backup (named Backup) so it runs a custom script before running:

--run-script-before=/home/me/[...]/Duplicati/dupicati-before-script.sh

Here is where it is configured on the UI:

image
image1343Ă—697 49.7 KB

And here is what’s inside my custom script:

#!/bin/bash

# Export configuration to home directory so it is backed up too.
duplicati-server-util export Backup --overwrite --unencrypted --destination /home/me/[...]/Duplicati/

I’m fine with leaving the configuration JSON file unencrypted as the whole backup is itself encrypted, but your mileage may vary.

Cheers.

1 Like
7

I don’t have a lot more to offer besides the concerns that were raised back then, which are:

  • Sensitive information is spread across different destinations
  • You need most of the information to regain it

I am mostly worried about the first part. If you have a backup that is not encrypted and several that are encrypted, you risk exposing data from an unrelated backup with a convenience setting.

The second point is perhaps incorrect in some usecases.

Yes, I think this is because the feature was never fully designed, so it only shows up on the restore page. We should perhaps remove it from there until it is fully developed.

I am very open to enabling the feature if there is a an actual use-case for it. I would like to have some idea for how we prevent users from mistakenly cross exposing backups though.

Great that you have something that works!

Generally though, I see two downsides to that approach:

  • You leave the file unencrypted on the local filesystem.
    I guess you could remove it as part of the shutdown script, but that is not robust with regards to crashes.
  • It is hard to recover it directly.
    You need to run a restore and extract the file.

Maybe a better solution overall is to integrate with an external secure storage, and then update the configurations there?

8

True, but that’s the case for all my personal files on my computer. What’s important to me is that it is encrypted on the remote storage backup server because it’s shared with other people, which is the case thanks to Duplicati.

The exported configuration json file does contain some sensitive info like my username and ssh fingerprint which is far from ideal, but at least it does not include the password.

But that’s because my approach is quite hacky, if it was a Duplicati feature, we should probably exclude those. And it could be encrypted.

True also, but again, I feel like that’s mainly because it’s a workaround! If Duplicati had the option to recover it directly from a remote backup (like the UI suggests), hopefully it would know how to directly extract it I guess.

I must say I’m not following that part. My backup is uploaded to an SFTP, so it’s secured in a sense, but that’s probably not what you meant… ?