I get the “Found xx remote files that are not recorded, please run repair” error, because the backup-job database is not moved. So instead of reading the sqlite file per backup-job from “systemprofile”-folder, it continues to read from my local user folder instead. (And when I moved the sqlite-files, it recreates empty ones.)
So my question is what steps are necessary to migrate the backup-sqlite files?
Hey, one small note.
After the migration Duplicati can’t find %my_documents%, %desktop% etc. source folders (if they were included in the backup) because it’s looking for them under %userprofile% path which for the service points to \Windows\system32\config…
Are these instructions now outdated? I followed them to the letter, and have ended up with a huge mess now. FWIW, I’m a sys admin, and follow(/create) instructions like these on a daily basis. This is not challenging stuff.
The real problem is that duplicati apparently no longer recognizes my database. I’ve got about 300GB of files that I spent the last several days uploading to B2, and it looks like I’m going to have to start from scratch. It gives me the error about finding 11XXX files not recorded in local storage. I checked to make sure it’s pointing to the correct path (it is - I’m looking at the DB in that path). I tried to repair the DB per the error message’s instructions, but as soon as I click the “repair” button under the DB menu, it just takes me back to the home screen with no indication that a repair is being attempted.
My first thought is to wipe out duplicati and reinstall from scratch, but even if I export my settings, I’m kind of doubtful that will allow me to not have to start my backup all over. Any suggestions?
Some other notes about the questions that are likely to be asked (or steps that are confusing):
I did close the tray icon, and all duplicati processes, before beginning. But the tray icon continues to come back. It’s not in the startup menu. Does it automatically start every time the service is started, or if you run Duplicati from the start menu? That should be mentioned.
The steps listed here about moving the database seem to be very wrong, based on later comments. The portion about src and dst is missing a few words, which makes it very confusing. But it seems like the correct way to do this (if it’s even truly needed) would be using the “Move existing database” feature? The DB move seems to be the root of most of the rest of my problems.
Port 8300 works, but port 8200 tries to go through the first run setup process. As I said, the tray icon was closed. Is 8300 vs 8200 a problem? No real details of the difference given here.
The initial post should be a wiki so anybody can edit it, but as I recal when the src / dst comment was made I went and updated the post as suggested.
I suppose in theory you could use the built in “Move existing database” feature but you’d need to do something like:
shift to service so you know where the database files will be stored
run the tray-icon using the old setting (so NOT attached to the new service server)
use the “Move existing database” feature to move from the tray-icon based folder to the service based folder
But I think that still won’t bring over your jobs and other settings.
It’s not a problem, but it can definitely be confusing. When Duplicati starts it first tries to grab port 8200. If that’s not available, then it increments by 100 and tries again (so port 8300) and repeats as necessary until an open port is found (so next would port 8400, 8500, etc.).
Normally a service starts before any users have logged in so usually if you have Duplicati running as a service it will be at port 8200 (this can be fixed / specified with a parameter if needed) and uses the service based storage locations (such as stuff in C:\Windows…). When the user logs in and runs the tray-icon it starts it’s own backend server which grabs the next port (in this case 8300) and uses the user based storage locations (such as C:\users\[your user name]\Roaming\Duplicati…")
I agree - I’d suggest starting by doing a system search for *.sqlite files to see if we can find your “misssing” database and get it put into the correct location.
@JonMikelV Great guide, but putting it in C:\Windows is possibly dangerous as the Windows directory now gets recreated when you update Windows. I just updated to the Windows 10 April 2018 update and Duplicati could no longer find my backups! I guessed what the problem was and copied the files over from the C:\Windows.old directory manually and it seems to be fine
Yeah, we know about the larger Windows updates not recreating everything as it should.
Unfortunately, that’s the folder Windows uses by default for the localhost user.
I know of at least one suggestion for Duplicati to check Windows.old if it finds absolutely NO config stuff, but so far it’s been a relatively infrequently occurring issue so it’s not high on the priority list.
But as more users move to service based usage that may change, particularly if Microsoft keeps jacking up the Windows folder during updates.
When running Duplicati in portable mode, the data is stored in a subfolder Data, located under the program folder (usually C:\Program files\Duplicati 2).
One exception is the location of the files that are downloaded by the integrated updater, discussed here:
More info about the problem with missing config after major Windows 10 updates:
New user here. This whole service installation sounds horribly hacky, unreliable, and non-robust, tbh. Not to mention updating seems like it would be problematic. The best solution, IMO is for “install as service” to be added as an option by the developers at the installer itself. For an excellent example of how this is done, see the Resilio Sync* Windows installer. Best of all, with Resilio Sync you can transition from tray icon to service via the installer. Just run it again and select the service option.
*I’m not implying Resilio is a competitor, because it isn’t. Just giving an example of a Windows application that does this right.
Thanks for your input & example about service installs. A feature like you’ve described has been discussed (see below), but so far development focus has been on performance and reliability improvements over installation (and even UI) experience.
I expect we’ll get to it at some point, but if you’d like to see it sooner than later and you, or somebody you know, has experience working with installers (or wants to gain some without the normal corporate time pressure) feel free to take a look at the code over at GitHub - duplicati/duplicati: Store securely encrypted backups in the cloud! and add the feature you think is missing.
I ran into some of the same issues others had noted and after running the process on 3 different systems (converting them all over to a service after they’d been running as a user app for awhile) came up with a similar but slightly different process that seems to work a little smoother (eg: without running into missing files / missing db errors).
So, plagiarizing @JonMikelV’s original comment (with changes)…
Here we’re going to go over method #1.
Stop the tray icon if you currently have it running
Create a folder called Duplicati in C:\Windows\System32\config\systemprofile\AppData\Local\. So you end up with C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati
** If you are prompted to give yourself access when accessing folders in C:\Windows\System32\, confirm that you want to continue
Move the file C:\Users\[your user name]\AppData\Local\Duplicati\Duplicati-server.sqlite to the folder you created in step 1, so that you end up with C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati\Duplicati-server.sqlite
**If you were running the portable version, your “from” location will be C:\Program Files\Duplicati 2\Data\Duplicati-server.sqlite instead
Open a command prompt as an administrator (right click and select “Run as administrator”) and run Duplicati.WindowService.exe install in your Duplicati install folder (likely "C:\Program Files\Duplicati 2\Duplicati.WindowsService.exe" install )
If you get the “Installation has completed” message, you can start the service with sc start Duplicati (or use a Windows Service GUI, your choice)
But if you get an error such as the following then you probably didn’t open your command prompt as an authorized administrator, try again System.UnauthorizedAccessException: Access to the path 'C:\Program Files\Duplicati 2\Duplicati.WindowsService.InstallState' is denied.
Once the service has been started go ahead and close the command prompt window (we’re done with it now)
Open a browser window and go to http://localhost:8200 to make sure the service is working and found your existing settings (if that fails but http://localhost:8300 works, then you may have forgotten to stop the tray icon)
Use the “Move existing database” feature for each existing backup set
8.1 Click on one of your backup sets to open the menu
8.2 Click on “Database” under the “Advanced” heading
8.3 Under the “Location” section, click in the “Local database path” field
8.4 Replace the C:\Users\ [your user name]\AppData\Local\Duplicati\ part with C:\Windows\System32\config\systemprofile\AppData\Local\, leaving the XXXXXXXXXX.sqlite part alone. So you’ll end up with something like C:\Windows\System32\config\systemprofile\AppData\Local\ABCDEFGHIJ.sqlite
8.5 If this backup set has already run at some point, click “Move existing database”. If the move button is disabled because the backup hasn’t run yet, click the “Save” button instead.
8.6 Repeat step 8.1 to 8.5 for any other backup sets you have
The Tray icon is no longer needed so you can remove Duplicati from your Startup programs (on my system this was found in C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp). However, if you like having the icon around then be sure to update the properties of any shortcuts to it (such as in the Startup folder) to include --no-hosted-server parameter** as part of the Target field (this tells the Tray Icon to connect to the service instead of starting it’s own instance).
** Note there’s an outstanding issue where the tray icon doesn’t work with this parameter. You can still set it up and try it, and when the bug is fixed it should start working again.
There you’re done! Duplicati should now run when the machine starts up whether or not a user is actually logged on.