It would be worth checking Task Manager if there is any chance that there are multiple Duplicati.
Having said that, I started 2.1.0.2 then 2.1.0.3, got a tray icon for each, and pausing one showed pause in both. I took a guess that 2.1.0.3 was the lower icon, so did right-click open. Got a hang. Thinking maybe I got 2.1.0.2 by mistake, I tried the other one. Got a hang. Task Manager shows Image path name column, so I know what was up. I also did manual launch from two zip installs.
Each hang leaves a remnant on the task bar, and a mis-shapen leftover of the system tray menu.
I did a Right-click Close window on the first taskbar Duplicati, thought to be 2.1.0.3, and 2.1.0,3 vanished from Task Manager. I did have to force it down from popup saying it wasn’t responding.
The fix is not explicitly mentioned in the 2.1.0.3 summary of changes since 2.1.0.2, but I thought I tested it before and it seemed to work. Regardless, now it doesn’t even on a fresh TrayIcon start. Windows service side is still 2.1.0.2 if it matters.