Sounds like some sort of version scramble. 2.0.3.2 would have known database version 5 but seen 6.
Installed WindowsService, decided I didn’t like it, tried to go back to Duplicati.GUI.TrayIcon.exe, now crashes on startup is the sort of duplicati-crashlog.txt it gets, except the 4 would be a 5 for your case.
But two questions would be why you don’t find it, and why there are still processes up, after DB error.
Downgrading / reverting to a lower version shows where updates are kept for different ways of install:
Windows
a.C:\ProgramData\Duplicati\updates(most Windows service instances)
b.C:\Users\<Duplicati user account>\AppData\Local\Duplicati\updates\(most Windows user / tray-icon instances)
so one might think it left a non-service-install update behind in your user profile, except you say it was alaready where it should be for a service (are there dates on files to show how long it was installed?).
If the problem was still sitting there, one can use Task Manager to check process Properties to check version information by the path to the executable, but it’s too late now. Regardless, I’m glad you’re up.
The renamed Duplicati-server.sqlite might have the old job info in it, but you already set up a new one.