Yeah, I think service vs. Tray Icon vs. logged in user is the most common cause of this issue. I almost feel there should be a check for other Duplicati instances so we can provide links on the GUI to the other ones.
Agreed. But this particular problem is not caused by Duplicati, but Microsoft. When upgrading Windows 10, the profile folder of the SYSTEM PROFILE account is moved to the Windows.old directory. The upgrade process should at least make a copy of this folder, not moving it out of the Windows folder. There’s nothing Duplicati can do about this, this should be fixed by Microsoft.
“Should”, yes. But while they are busy taking care of that issue , I suggest that duplicati becomes more robust. As ridiculous as it sounds (is!), if there is a realistic chance that the OS will pull the floor away underneath your feet, you need to make sure you check the floor is still there at startup. If it’s not, you issue an error message.
@kenkendk is aware of the problem, but I’m not sure it’s on top of his priority list.
Just to keep this fresh I thought I’d mention one of my machines just got the update and copying the Duplicati folder from / to the following folders worked for me as well, however I did need to “grant myself access” to the C:\windows folders (in other words Windows would would say “you don’t have access, click here to grant yourself access”):
Since this seems to be a fix I went ahead and flagged the appropriate post as the solution (in case others come looking).
And this happens also with the October update. Remediation presented here works fine.
Thanks for letting us know. I believe this is now in the Todo list, though exactly how and when I don’t yet know.
I confirm the same behaviour with 1809, good old Microsoft, but so far it’s the only issue I’ve had with 3 different Windows 10 machines I have upgraded.
I also agree that the application should use “ProgramData” instead of the profile when run as a service, it’s how many other service-based products work, the Duplicati folder under the System profile was the only non-Microsoft data I have found.
Strange is, that this behavior of storing files in C:\Windows\System32\config\systemprofile is also used in other backup SW, for Example in Veeam.
(Article is from 2015 - so they don’t have any motivation to fix it)
All the more reason not to use Veeam then
It’s most certainly not the way Veritas System Recovery and Backup Exec works (and under Symantec before that).
When I started using Duplicati it was the first product I have come across that used that folder, on my 3 servers and two workstations none of them have anything other than Microsoft folders in their, although I did find Java on one when fixing this issue, but that was uninstalled years ago.
From newb’s post above
Apparently windows.old automatically gets deleted after a month
I just confirmed in the latest version 1809 update that it’s only 10 days that you can fall back to the previous version before it’s deleted.
I had to re-import my backup profiles, run database recovery on each profile, and configure all my global settings to get it back up.
Is everyone using Duplicati to actually backup the Windows\System32\config\systemprofile\AppData\Duplicati folders as well? I haven’t tried this yet.
I wouldn’t have thought this would work very well, I just remember to make a copy of it before any major upgrade of Windows.
Having Duplicati back up its own databases is tricky because they can be either locked (which –snapshot-policy can help) or not yet done (which --snapshot-policy can’t fix but having jobs back each other up can).
The SYSTEM profile is a bit more secure against users (but not Microsoft). It protects Duplicati better (the passwords, for example), but my databases are under ProgramData, with Security set to my liking (open)
Migrating from User to Service install on Windows is my input to a similar discussion to change the default rather than have Duplicati get wiped away twice a year. I’m not sure if it’s controversial, or just not yet done.
This isn’t a “fix” of course (remember, this is working as designed so it’s not a bug) but one option would be to use portable mode.
Activates portable mode where the database is placed below the program executable.
And of course specifying the database folders would probably get the same result:
Duplicati needs to store a small database with all settings. Use this option to choose where the settings are stored. This option can also be set with the environment variable DUPLICATI_HOME.
Note that while I’ve used these in the past I didn’t verify it resulted in nothing going to the system folder so YMMV.
Perhaps so, but see above. Designs sometimes don’t have complete information at the time of design:
I completely agree - and there’s nothing to say Microsoft won’t go and start fiddling with ProgramData in their next release so it’s definitely worth addressing if possible.
I guess I was just trying to point out that a “oh,b it’s not supposed to do that” bug is likely to get higher priority than a “that’s s known side effect of our design choice” issue.
I know there’s a long discussion about this elsewhere (night even be the link you provided) that had some reasons for not using ProgramData. Though if I recall correctly they were painful than this issue happening now.
I assume Linux based installs don’t run into a similar issue because Duplicati is running on a sensibly designed OS.
Is this fixed in newer versions of duplicati?
I see @kees-z is saying this is a Microsoft problem and they should fix it, but this problem still happens when windows updates in 2020.
Duplicati’s behavior has not changed yet. The recommendation for now is to make sure you store your data outside of C:\Windows if you opt to run as a service tied to the LocalSystem account.
A future version of Duplicati should hopefully default somewhere else, probably C:\ProgramData.
For the time being, you can avoid this by supplying
Duplicati.WindowsService.exe. This will store all data (job definitions, local databases) outside the LocalSystem profile folder. A subfolder
data of the Duplicati program folder is used instead, leaving the
Windows folder untouched.
We can work around this. In most cases, including in particular when Duplicati runs as SYSTEM, it can configure custom directory permissions. For example, it can change permissions such that ordinary users cannot access its directory. Microsoft SQL Server does this to prevent users from accessing database files directly (see File System Permissions Granted to SQL Server Per-service SIDs or Local Windows Groups).