I just found this on Github: Configuration "lost" after Windows 10 Anniversary Update when running as a service · Issue #1950 · duplicati/duplicati · GitHub
I managed to find my Duplicati settings under C:\Windows.old\Windows\System32\config\systemprofile\AppData\Local\Duplicati so I moved them over to C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati and that seems to fix things.
It looks like the Fall Creators Update decided to wipe out all of the settings (but luckily kept a backup)