Looks like an issue is open for this bug:
1 Like
Thanks for the better search than I did. I put a note on it with the suggestion here, and sought volunteers.
Having the default result be loss of configuration and databases every Windows version update is bad…
1 Like
I should emphasize that I don’t really care why Duplicati tends to lose its configuration on a regular basis (it’s happened many times across a couple of computers). I just want some better options for dealing with it than having to remember to manually export my configuration on a regular basis, having to dig through obscure system folders, etc. Or, ideally, to have Duplicati be a lot more resilient, because it’s really bad that it keeps jettisoning its configuration multiple times per year.
Is there some command line utility I can run to cause it to create a JSON file the way I can from the browser UI? This would allow me to build a script that I can run as a periodic task.
1 Like
Puzzling, why don’t you care about the root issue? Are you running it as a service, as mentioned in earlier replies? I run Duplicati on many machines and never have this issue because in the cases where I run it as a service I have the data stored elsewhere.
In any case, if you want to just automate the export I believe the excellent duplicati-client program can do it (it has an export command). But I haven’t tried it myself.
You could run it automatically using –run-script-after. Alternatively (or in conjunction) you could also copy the sqlite databases to another location in this same script.
I don’t care about the root issue because as an end user it shouldn’t be my problem, and because it happens so much that having to dig around for lost config files is not anything approaching a reasonable solution.
I’m absolutely running Duplicati as a local system account service. I have no idea how that’s supposed to magically address anything, as I’m obviously still encountering the lost configuration issue on a regular basis.
Anyway, after striking out with the official command line utilities, I was able to use my web browser’s developer tools to extract the URL needed to export a JSON configuration. I’ll probably create a task to grab this via CURL or whatever, although I’ll have to also make it recognize when it’s reached the inevitable state where Duplicati has once again lost its config data, or else I might end up wiping out my config backup as well.
Well, you are using a non-default setup by running it as a service. That’s exactly why you’re having this problem in the first place. Since you already deviated from the default, you could do a bit more customization to avoid this problem entirely.
The service was installed via Duplicati’s own command line tool via Duplicati’s own instructions. If that results in a broken configuration, then it’s also a Duplicati issue and not my fault, or else it shouldn’t provide that option.
The documentation basically encourages running it as a service, and makes no suggestion that it’s unsupported: Installation - Duplicati 2 User’s Manual
The service installation instructions also mention absolutely nothing about customization or storing configuration in alternate locations, so that sounds to me like a non-default way to go about running Duplicati as a service: Other Command Line Utilities - Duplicati 2 User’s Manual
This situation is not my fault, period. Stop trying to shift the blame onto me, please, it’s not helpful.
I’m trying to help you! I’m not one of the core Duplicati developers at all, just a volunteer on the forum. Running as a service is supported (it is not the default setup), but the current standard behavior puts the data in a location that is at risk when you do major Windows updates. This is something that should be fixed in Duplicati. But you can easily avoid this problem. Please see earlier posts.
I run Duplicati as a service and also with datafolder set to C:\ProgramData, i.e. I installed the service as
Duplicati.WindowsService.exe install --server-datafolder C:\ProgramData\Duplicati
Nonetheless I also lost all backup configurations upon Windows11 update!
In fact, checking now, I see some backup data is really kept in C:\ProgramData, but some other data is still kept in C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati (namely file Duplicati-server.sqlite and directory control_dir_v2).
I there any way to tell duplicati service to keep everything in C:\ProgramData\Duplicati ?
Can you confirm that the --server-datafolder customization was actually applied to the service? Open the service properties and check the “path to executable” line (you may need to scroll that line to the right…)
Yes, applied as confirmed by service properties:
In fact I correclty see one .sqlite file for each of my backup jobs in C:\ProgramData\Duplicati, plus an “updates” folder. Still Duplicati is keeping some other data in \windows\system32…, probably the main .sqlite config file and I also see this in Duplicati web page → About → System info (note the Path in SpecialFolders for %APPDATA% and %HOME%):
ServerVersion : 2.0.6.3
CLRVersion : 4.0.30319.42000
CLROSInfo : {"Platform":"Win32NT","ServicePack":"","Version":"10.0.22000.0","VersionString":"Microsoft Windows NT 10.0.22000.0"}
ServerModules : []
UsingAlternateUpdateURLs : false
LogLevels : ["ExplicitOnly","Profiling","Verbose","Retry","Information","DryRun","Warning","Error"]
SuppressDonationMessages : false
SpecialFolders : [{"ID":"%APPDATA%","Path":"C:\\WINDOWS\\system32\\config\\systemprofile\\AppData\\Roaming"},{"ID":"%HOME%","Path":"C:\\WINDOWS\\system32\\config\\systemprofile"}]
The option syntax seems to be option=value (not option value), confirmed by some test.
But the most important thing is to get your old config out of Windows.old before it goes away.
EDIT:
I think these are based on the user. You’re not trying to change that, just change DB location.
Perhaps you did a manual migration of those, per directions on how to migrate to a service?
Regardless, if that part is fine, stop the service, uninstall, reinstall with correct option syntax.
Either put your old duplicati-server.sqlite there, start service, and see if it comes up properly,
or if old configs have aged away then start service and import exported configs (if available).
1 Like
Interesting, thanks for catching this. I’ll try that and will update with the outcome.
Yes, yes. Already restored from Windows.old. I’m just looking for a fix so that I don’t have to do it again and again in the future…
Great, this worked. Now everything seems to correclty be in C:\ProgramData\Duplicati for me (and I have been able to just move the files from \windows\system32…\Duplicati, so all job configuration has been preserved).
Just one last doubt, I use –server-datafolder=“C:\ProgramData\Duplicati”, but I see someone suggests –server-datafolder=“C:\ProgramData\Duplicati\Data” instead (e.g. here). Is there any good reason you know of to keep everything one level deeper?
1 Like
I can’t think of much. The main need is to get it out of SYSTEM AppData which Windows upgrade clears.
NTFS folder permissions could, I suppose, be tightened for a subfolder that only SYSTEM has to get into. There might be data for other users of that system, destination information, etc. that should be restricted.
ProgramData is pretty open by default. I don’t know if multiple Duplicati users share upgrades (why not?).
From a style point of view, --server-datafolder might suggest a data folder called data, which is also what --portable-mode creates, but underneath the install folder, for example C:\Program Files\Duplicati 2.
Thanks everyone. I have run Duplicati.WindowsService.exe install --server-datafolder="%ProgramData%\Duplicati\data" and will see how it goes. I can confirm that it created a data folder in the expected path. Not sure why this isn’t the default behavior for system service mode.
Possibly the original developer wasn’t aware that Windows had this unfortunate behavior.
The current behavior is entirely consistent with the location for a normal user account, i.e.
C:\>whoami
hp4\maintenance
C:\>echo %LOCALAPPDATA%
C:\Users\Maintenance\AppData\Local
C:\>whoami
nt authority\system
C:\>echo %LOCALAPPDATA%
C:\Windows\system32\config\systemprofile\AppData\Local
but it’s a bad spot, if Windows keeps deleting it. There’s an open issue discussing changing:
Configuration “lost” after Windows 10 Anniversary Update when running as a service #1950
Addendum: I was also able to migrate my existing settings on a machine that already had Duplicati installed in service mode. I stopped + uninstalled the service, moved the data files to the new location, and reinstalled+restarted the service with the parameter that points it to the new location.
Unfortunately it then went a few days without backing up until I noticed. Turns out you have to also update the local database path in the web UI, or else it tries to access it in the old location and reports a locked file error (?).
if you export a job as command line, you’ll see in the --dbpath parameter that the server database stores an absolute path to the job database. In practice by default, the job databases are in the same directory as the server database, but it’s not necessary. So moving all the databases together is not enough.
The people migrating from user to service install found probably the same thing in v2 below.
It looks like v3 found out the risk of the current default location. So maybe you got both now.