Duplicati seems to lose its configuration whenever there’s a slight breeze. It’s happened to me many times, most recently as an apparent side effect of an in-place upgrade from Windows 10 to Windows 11.
I’ve started doing occasional manual exports of my configurations to JSON files on my backup server, but given how much of a problem this has been for me, it sure would be helpful if there were a feature to auto-export/backup its own configuration etc. on a schedule (or on change) as well.
Edit: I’m not sure if I can actually even blame the Windows 11 upgrade, because I just checked on my Windows 10 machine and it lost its configuration at some point as well haha. This is such a huge issue with Duplicate for me. It’s just not something you can set and forget for more than a month, because it will eventually lose its marbles.
Are you running Duplicati as a service under the special LocalSystem user context?
Duplicati defaults to storing its configuration in the user profile. For LocalSystem this is C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati. This area is unfortunately affected by major Windows 10 updates (like upgrading from 20H2 to 21H1), and presumably when you upgrade from Windows 10 to Windows 11.
After an upgrade you can find the Duplicati files in C:\Windows.old. You could copy them back to the new C:\Windows folder, but the better option is to not store those files there at all.
(In my opinion Duplicati should store the data in C:\ProgramData when it’s being run as LocalSystem.)
There are many previous threads on this topic, which you can peruse for more info. Here are a couple:
Windows 10 version upgrades can also do this, although I think some of the recent ones were just feature enablers so might not have done the whole Windows.old thing which sweeps Duplicati config + DB away.
That’s my preferred location to stop Windows wipe-out, although others were proposed. Anything’s better. Although it might not get any action soon, do you know if there’s a GitHub issue filed asking for the move? One developer-volunteer challenge is that we might be low on developers running Windows these days…
Thanks for the better search than I did. I put a note on it with the suggestion here, and sought volunteers.
Having the default result be loss of configuration and databases every Windows version update is bad…
I should emphasize that I don’t really care why Duplicati tends to lose its configuration on a regular basis (it’s happened many times across a couple of computers). I just want some better options for dealing with it than having to remember to manually export my configuration on a regular basis, having to dig through obscure system folders, etc. Or, ideally, to have Duplicati be a lot more resilient, because it’s really bad that it keeps jettisoning its configuration multiple times per year.
Is there some command line utility I can run to cause it to create a JSON file the way I can from the browser UI? This would allow me to build a script that I can run as a periodic task.
Puzzling, why don’t you care about the root issue? Are you running it as a service, as mentioned in earlier replies? I run Duplicati on many machines and never have this issue because in the cases where I run it as a service I have the data stored elsewhere.
In any case, if you want to just automate the export I believe the excellent duplicati-client program can do it (it has an export command). But I haven’t tried it myself.
You could run it automatically using –run-script-after. Alternatively (or in conjunction) you could also copy the sqlite databases to another location in this same script.
I don’t care about the root issue because as an end user it shouldn’t be my problem, and because it happens so much that having to dig around for lost config files is not anything approaching a reasonable solution.
I’m absolutely running Duplicati as a local system account service. I have no idea how that’s supposed to magically address anything, as I’m obviously still encountering the lost configuration issue on a regular basis.
Anyway, after striking out with the official command line utilities, I was able to use my web browser’s developer tools to extract the URL needed to export a JSON configuration. I’ll probably create a task to grab this via CURL or whatever, although I’ll have to also make it recognize when it’s reached the inevitable state where Duplicati has once again lost its config data, or else I might end up wiping out my config backup as well.
Well, you are using a non-default setup by running it as a service. That’s exactly why you’re having this problem in the first place. Since you already deviated from the default, you could do a bit more customization to avoid this problem entirely.
The service was installed via Duplicati’s own command line tool via Duplicati’s own instructions. If that results in a broken configuration, then it’s also a Duplicati issue and not my fault, or else it shouldn’t provide that option.
The service installation instructions also mention absolutely nothing about customization or storing configuration in alternate locations, so that sounds to me like a non-default way to go about running Duplicati as a service: Other Command Line Utilities - Duplicati 2 User’s Manual
This situation is not my fault, period. Stop trying to shift the blame onto me, please, it’s not helpful.
I’m trying to help you! I’m not one of the core Duplicati developers at all, just a volunteer on the forum. Running as a service is supported (it is not the default setup), but the current standard behavior puts the data in a location that is at risk when you do major Windows updates. This is something that should be fixed in Duplicati. But you can easily avoid this problem. Please see earlier posts.
I run Duplicati as a service and also with datafolder set to C:\ProgramData, i.e. I installed the service as Duplicati.WindowsService.exe install --server-datafolder C:\ProgramData\Duplicati
Nonetheless I also lost all backup configurations upon Windows11 update!
In fact, checking now, I see some backup data is really kept in C:\ProgramData, but some other data is still kept in C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati (namely file Duplicati-server.sqlite and directory control_dir_v2).
I there any way to tell duplicati service to keep everything in C:\ProgramData\Duplicati ?
Can you confirm that the --server-datafolder customization was actually applied to the service? Open the service properties and check the “path to executable” line (you may need to scroll that line to the right…)
Yes, applied as confirmed by service properties:
In fact I correclty see one .sqlite file for each of my backup jobs in C:\ProgramData\Duplicati, plus an “updates” folder. Still Duplicati is keeping some other data in \windows\system32…, probably the main .sqlite config file and I also see this in Duplicati web page → About → System info (note the Path in SpecialFolders for %APPDATA% and %HOME%):
The option syntax seems to be option=value (not option value), confirmed by some test.
But the most important thing is to get your old config out of Windows.old before it goes away.
EDIT:
I think these are based on the user. You’re not trying to change that, just change DB location.
Perhaps you did a manual migration of those, per directions on how to migrate to a service?
Regardless, if that part is fine, stop the service, uninstall, reinstall with correct option syntax.
Either put your old duplicati-server.sqlite there, start service, and see if it comes up properly,
or if old configs have aged away then start service and import exported configs (if available).
Great, this worked. Now everything seems to correclty be in C:\ProgramData\Duplicati for me (and I have been able to just move the files from \windows\system32.…\Duplicati, so all job configuration has been preserved).
Just one last doubt, I use –server-datafolder=“C:\ProgramData\Duplicati”, but I see someone suggests –server-datafolder=“C:\ProgramData\Duplicati\Data” instead (e.g. here). Is there any good reason you know of to keep everything one level deeper?
I can’t think of much. The main need is to get it out of SYSTEM AppData which Windows upgrade clears.
NTFS folder permissions could, I suppose, be tightened for a subfolder that only SYSTEM has to get into. There might be data for other users of that system, destination information, etc. that should be restricted.
ProgramData is pretty open by default. I don’t know if multiple Duplicati users share upgrades (why not?).
From a style point of view, --server-datafolder might suggest a data folder called data, which is also what --portable-mode creates, but underneath the install folder, for example C:\Program Files\Duplicati 2.
Thanks everyone. I have run Duplicati.WindowsService.exe install --server-datafolder="%ProgramData%\Duplicati\data" and will see how it goes. I can confirm that it created a data folder in the expected path. Not sure why this isn’t the default behavior for system service mode.
Possibly the original developer wasn’t aware that Windows had this unfortunate behavior.
The current behavior is entirely consistent with the location for a normal user account, i.e.
