I installed the Windows 10 Fall Creators Update yesterday on my computer. I noticed that when I open Duplicati, it no longer shows any of my backup tasks, and the options have been reset to default. I found this in the log:
Oct 18, 2017 6:39 PM: Request for http://localhost:8200/api/v1/serverstate?longpoll=true&lasteventid=15&format=json&duration=5m gave error
System.Threading.ThreadAbortException: Thread was being aborted.
at System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, UInt32 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
at Duplicati.Server.EventPollNotify.Wait(Int64 eventId, Int32 milliseconds)
at Duplicati.Server.WebServer.RESTMethods.RequestInfo.LongPollCheck(EventPollNotify poller, Int64& id, Boolean& isError)
at Duplicati.Server.WebServer.RESTMethods.ServerState.GET(String key, RequestInfo info)
at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
Oct 18, 2017 6:39 PM: Reporting error gave error
System.ObjectDisposedException: Cannot write to a closed TextWriter.
at System.IO.__Error.WriterClosed()
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
Iâm running Duplicati as a service (localhost:8200) if that makes a difference. Any idea what is going on?
I managed to find my Duplicati settings under C:\Windows.old\Windows\System32\config\systemprofile\AppData\Local\Duplicati so I moved them over to C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati and that seems to fix things.
It looks like the Fall Creators Update decided to wipe out all of the settings (but luckily kept a backup)
Sorry if this is covered in they linked article⌠When running Duplicati as a service settings are stored in the âC:\Windows\System32\config\systemprofile\AppData\Local\Duplicatiâ folder by default (unless certain environment variables or parameters are set).
Certain Windows updates (or upgrades to Windows 10) back up the entire C:\Windows folder and install into a new one (then bring over the settings they care about from the old one).
Iâm not sure thereâs much that can be done for this other than documenting it or maybe having Duplicati look for a Windows.old folder if the regular Windows folder has no settings.
Wouldnât it make more sense to store the settings in another folder instead? It doesnât seem like the best idea to store important settings in a folder that Windows will be wiping out regularly (it seems like Microsoft is doing two âbigâ updates every year).
Apparently windows.old automatically gets deleted after a month. If people arenât regularly checking Duplicati, that could be a problem.
I fully agree, but keep in mind that this is more an issue hat Microsoft should solve than a bug in Duplicati.
Duplicati stores all settings and local DBâs in the AppData folder under the userâs profile. Most likely this is a subfolder of C:\User\<Username>.
When Duplicati is registered as a Windows service, it will be started using the Local System account. This is not a ârealâ user account, you canât log on interactively with it. This account also has a profile folder, but it is not located in C:\Users.
The location used to store the Local Systemâs profile is C:\Windows\System32\config\systemprofile. The environment variable %APPDATA% (this variable is used by Duplicati to choose its storage location) points to C:\Windows\System32\config\systemprofile\AppData.
This is why Duplicati DB files are stored in C:\Windows\System32\config\systemprofile\AppData\Duplicati if it is started as a service.
During a Windows 10 upgrade, the complete Windows folder is renamed to Windows.old. The installer tells that all user data will be retained, but it seems the Local Systemâs profile folder is forgotten. The contents of the systemprofile are not part of Windows, but contain most likely user data. So Microsoft should fix this.
There are a few workarounds you can use until Microsoft has fixed this in the Windows 10 Upgrade procedure:
Run the service in Portable Mode:
Unregister the service with Duplicati.WindowsService.exe uninstall and re-register it with Duplicati.WindowsService.exe install --portable-mode.
Duplicati will use the folder named Data under the program folder. Copy or move the contents from C:\Windows\System32\config\systemprofile\AppData\Duplicati to the Data folder under the Duplicati program folder (most likely C:\Program Files\Duplicati 2).
Specify a custom data folder:
Unregister the service with Duplicati.WindowsService.exe uninstall and re-register it with Duplicati.WindowsService.exe install --server-datafolder=<path>.
Almost the same as the first workaround, but you can specify a location outside the Duplicati program folder.
Start the Windows service with a ârealâ user account, preferably an account with administrative privileges. This will move the Duplicati data folder from the systemprofile folder to the userâs AppData folder, located under `C:\Users. Again, copy or move the contents of the Duplicati folder from the old to the new location.
Wow, thanks for the detailed response. I now understand why this is happening. Those workarounds seem perfectly reasonable to me, I will implement one of them when I get time tomorrow.
Thanks for pointing that out. Existing backup jobs need to be modified to use the new path to the local DBâs.
If Duplicati could somehow test if the Duplicati Server instance is started with the SYSTEM account, it could change the storage location from %APPDATA%/%LOCALAPPDATA% to %PROGRAMDATA%.
Not sure if there could be any security issues, %PROGRAMDATA% contents are accessible for all users, the systemprofile isnât. Hopefully there are no big issues with that, because --portable-mode has the same effect.
Note that using %PROGRAMDATA% is also discussed in GitHib issue 2222 and 2826, maybe these can be combined in a single fix.
I am SO GRATEFUL I found this thread! I had a Windows update a couple of days ago, then started noticing my backups werenât working. Didnât connect the two events until I went to investigate tonight and found everything was gone! Started panicking until I found this thread and was back up and running in 5 minutes. THANK YOU ALL for the research on this!
Yeah, I think service vs. Tray Icon vs. logged in user is the most common cause of this issue. I almost feel there should be a check for other Duplicati instances so we can provide links on the GUI to the other ones.
Agreed. But this particular problem is not caused by Duplicati, but Microsoft. When upgrading Windows 10, the profile folder of the SYSTEM PROFILE account is moved to the Windows.old directory. The upgrade process should at least make a copy of this folder, not moving it out of the Windows folder. Thereâs nothing Duplicati can do about this, this should be fixed by Microsoft.
âShouldâ, yes. But while they are busy taking care of that issue , I suggest that duplicati becomes more robust. As ridiculous as it sounds (is!), if there is a realistic chance that the OS will pull the floor away underneath your feet, you need to make sure you check the floor is still there at startup. If itâs not, you issue an error message.
Just to keep this fresh I thought Iâd mention one of my machines just got the update and copying the Duplicati folder from / to the following folders worked for me as well, however I did need to âgrant myself accessâ to the C:\windows folders (in other words Windows would would say âyou donât have access, click here to grant yourself accessâ):
From C:\Windows.old\Windows\System32\config\systemprofile\AppData\Local\Duplicati
To C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati
Since this seems to be a fix I went ahead and flagged the appropriate post as the solution (in case others come looking).
I confirm the same behaviour with 1809, good old Microsoft, but so far itâs the only issue Iâve had with 3 different Windows 10 machines I have upgraded.
I also agree that the application should use âProgramDataâ instead of the profile when run as a service, itâs how many other service-based products work, the Duplicati folder under the System profile was the only non-Microsoft data I have found.
Strange is, that this behavior of storing files in C:\Windows\System32\config\systemprofile is also used in other backup SW, for Example in Veeam.
(Article is from 2015 - so they donât have any motivation to fix it)
Itâs most certainly not the way Veritas System Recovery and Backup Exec works (and under Symantec before that).
When I started using Duplicati it was the first product I have come across that used that folder, on my 3 servers and two workstations none of them have anything other than Microsoft folders in their, although I did find Java on one when fixing this issue, but that was uninstalled years ago.
, I suggest that duplicati becomes more robust. As ridiculous as it sounds (is!), if there is a realistic chance that the OS will pull the floor away underneath your feet, you need to make sure you check the floor is still there at startup. If itâs not, you issue an error message.
