That indicates that you are hitting two different databases. Duplicati stores the last-used port in the server database, and should keep using 8300 if it did once.
You can choose the port with --webservice-port=8200 on the commandline. You can change the password with something like:
sudo duplicati-server-util change-password
This should use the database in /root/.config/Duplicati/Duplicati-server.sqlite, which is also what the service should use (if running as root).
My guess is that you run the TrayIcon with your local user context, and that leads to ~/.config/Duplicati/Duplicati-server.sqlite.