Hi,
I managed to get rid of the autostarts of the TrayIcon. There was indeed a shortcut in the folder you mentioned and Autoruns showed me that I had also made an entry in the task scheduler (which I completely forgot about). It is amazing that Duplicati ran relatively smoothly, until some weeks ago, despite the fact that there were three different manners configured to start it.
The service is now running again and after moving the existing 2.0.6.3 folder to another location, I could download and activate the update again. This update now is surviving reboots.
Initially, the Home screen was empty, but after copying the profile from the local to that of the system user, I can now access my old backups and do no longer get that database version error.
That update folder in question was in the root of 2.0.6.3 and contained two files, installation.txt and README.txt.
Thanks for all your help!