Installed Windows 10 Fall Creators Update - Now Everything is Missing In Duplicati

planned
#7

Thanks for pointing this out, I was not aware that the system account is in C:\Windows, nor that the big updates actually removes the folder.

I think this warrants some special handling from Duplicati to avoid using folders that are inside the Windows folder.

3 Likes

#8

Don’t forget to move your databases, too.
Otherwise the existing backup jobs will stay in the systemprofile.

1 Like

#9

Thanks for pointing that out. Existing backup jobs need to be modified to use the new path to the local DB’s.

If Duplicati could somehow test if the Duplicati Server instance is started with the SYSTEM account, it could change the storage location from %APPDATA%/%LOCALAPPDATA% to %PROGRAMDATA%.
Not sure if there could be any security issues, %PROGRAMDATA% contents are accessible for all users, the systemprofile isn’t. Hopefully there are no big issues with that, because --portable-mode has the same effect.

Note that using %PROGRAMDATA% is also discussed in GitHib issue 2222 and 2826, maybe these can be combined in a single fix.

1 Like

#10

I am SO GRATEFUL I found this thread! I had a Windows update a couple of days ago, then started noticing my backups weren’t working. Didn’t connect the two events until I went to investigate tonight and found everything was gone! Started panicking until I found this thread and was back up and running in 5 minutes. THANK YOU ALL for the research on this!

0 Likes

#11

Yeah, I think service vs. Tray Icon vs. logged in user is the most common cause of this issue. I almost feel there should be a check for other Duplicati instances so we can provide links on the GUI to the other ones.

1 Like

#12

Agreed. But this particular problem is not caused by Duplicati, but Microsoft. When upgrading Windows 10, the profile folder of the SYSTEM PROFILE account is moved to the Windows.old directory. The upgrade process should at least make a copy of this folder, not moving it out of the Windows folder. There’s nothing Duplicati can do about this, this should be fixed by Microsoft.

0 Likes

#13

“Should”, yes. But while they are busy taking care of that issue :stuck_out_tongue_winking_eye:, I suggest that duplicati becomes more robust. As ridiculous as it sounds (is!), if there is a realistic chance that the OS will pull the floor away underneath your feet, you need to make sure you check the floor is still there at startup. If it’s not, you issue an error message.

1 Like

#14

@kenkendk is aware of the problem, but I’m not sure it’s on top of his priority list.

1 Like

#15

Just to keep this fresh I thought I’d mention one of my machines just got the update and copying the Duplicati folder from / to the following folders worked for me as well, however I did need to “grant myself access” to the C:\windows folders (in other words Windows would would say “you don’t have access, click here to grant yourself access”):

  • From C:\Windows.old\Windows\System32\config\systemprofile\AppData\Local\Duplicati
  • To C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati

Since this seems to be a fix I went ahead and flagged the appropriate post as the solution (in case others come looking).

1 Like

#16

And this happens also with the October update. Remediation presented here works fine.

0 Likes

#17

Thanks for letting us know. I believe this is now in the Todo list, though exactly how and when I don’t yet know. :slight_smile:

0 Likes

#18

I confirm the same behaviour with 1809, good old Microsoft, but so far it’s the only issue I’ve had with 3 different Windows 10 machines I have upgraded.

I also agree that the application should use “ProgramData” instead of the profile when run as a service, it’s how many other service-based products work, the Duplicati folder under the System profile was the only non-Microsoft data I have found.

0 Likes

#19

Strange is, that this behavior of storing files in C:\Windows\System32\config\systemprofile is also used in other backup SW, for Example in Veeam.
(Article is from 2015 - so they don’t have any motivation to fix it)

0 Likes

#20

All the more reason not to use Veeam then :wink:

It’s most certainly not the way Veritas System Recovery and Backup Exec works (and under Symantec before that).

When I started using Duplicati it was the first product I have come across that used that folder, on my 3 servers and two workstations none of them have anything other than Microsoft folders in their, although I did find Java on one when fixing this issue, but that was uninstalled years ago.

0 Likes

#21

From newb’s post above

Apparently windows.old automatically gets deleted after a month

I just confirmed in the latest version 1809 update that it’s only 10 days that you can fall back to the previous version before it’s deleted.

I had to re-import my backup profiles, run database recovery on each profile, and configure all my global settings to get it back up.

Is everyone using Duplicati to actually backup the Windows\System32\config\systemprofile\AppData\Duplicati folders as well? I haven’t tried this yet.

0 Likes

#22

I wouldn’t have thought this would work very well, I just remember to make a copy of it before any major upgrade of Windows.

0 Likes

#23

Having Duplicati back up its own databases is tricky because they can be either locked (which –snapshot-policy can help) or not yet done (which --snapshot-policy can’t fix but having jobs back each other up can).

The SYSTEM profile is a bit more secure against users (but not Microsoft). It protects Duplicati better (the passwords, for example), but my databases are under ProgramData, with Security set to my liking (open)

Migrating from User to Service install on Windows is my input to a similar discussion to change the default rather than have Duplicati get wiped away twice a year. I’m not sure if it’s controversial, or just not yet done.

0 Likes

#24

This isn’t a “fix” of course (remember, this is working as designed so it’s not a bug) but one option would be to use portable mode.

--portable-mode
Activates portable mode where the database is placed below the program executable.

And of course specifying the database folders would probably get the same result:

--server-datafolder
Duplicati needs to store a small database with all settings. Use this option to choose where the settings are stored. This option can also be set with the environment variable DUPLICATI_HOME.

Note that while I’ve used these in the past I didn’t verify it resulted in nothing going to the system folder so YMMV.

0 Likes

#25

Perhaps so, but see above. Designs sometimes don’t have complete information at the time of design:

0 Likes

#26

I completely agree - and there’s nothing to say Microsoft won’t go and start fiddling with ProgramData in their next release so it’s definitely worth addressing if possible.

I guess I was just trying to point out that a “oh,b it’s not supposed to do that” bug is likely to get higher priority than a “that’s s known side effect of our design choice” issue.

I know there’s a long discussion about this elsewhere (night even be the link you provided) that had some reasons for not using ProgramData. Though if I recall correctly they were painful than this issue happening now.

I assume Linux based installs don’t run into a similar issue because Duplicati is running on a sensibly designed OS. :slight_smile:

0 Likes