Ah - I see now on the original screenshot that the progress bar shows the “PC Drive D” backup is still running but the job details say the last SUCCESSFUL run was Tuesday.
So it sounds like the backups may be firing as they should be never finish for some reason - and aren’t logging anything about why? The UI almost needs to distinguish between last start time and last end time…
Does the progress bar count down as expected? I’m trying to figure out if the job is actually running but not logging…
Oh, and did you do anything silly like shift from client to service installs (or anything else that might have caused a change in where config or sqlite files would be stored)?
You didn’t mention what version of Duplicati you were running, but is it possible you’re running more than one instance (thus causing the locked database)?
Personally, I went from 184.108.40.206 beta client install to a service install then through 3 canary versions and on 220.127.116.11 things totally blew up and I had to uninstall Duplicati all together before a re-install would even let the GUI or service start up.
Through all those updates I think I found (but ultimately destroyed) an scenario of both the service running as well as the client - each with their own server instance. So even though I had it all working with the service at one point, an update (or something I did) split that out into using two different server runs.
Duplicati should handle connection interruptions, but after the connection is restored, some cleanup actions could be needed. Partially uploaded files, temporary files and files that are not in sync with the local DB should be detected and cleaned up and fixed. Turning off your computer during a running backup should be allowed, but is not recommended.
Having said that, I guess hiberbation could be a problem in combination with using some particular (or all?) backends. When putting your computer in hibernation, all processes continue after switching on the computer. This may cause conflicts with some or all types of backends.
The connection to the cloud provider probably timed out, so Duplicati has to reconnect and re-authenticate in the middle of an upload. If you shutdown your computer (instead of hibernating), all programs, including Duplicati, will be closed and the backup starts from scratch after powering on your computer.
I guess this could have something to do with the problem that your backup runs forever. After hibernation, connection to the cloud provider is lost while Duplicati is in the middle of an upload and notices that it’s time for a new scheduled backup. The new backup tries to start, but the database is still locked by the previous scheduled run, which will cause the “Database is locked” message.
This this is the most plausible explanation I can think of for problem that the database is locked and no new backup tasks will start.
I can reproduce the “Database is locked” message. Just throw a few hundreds MB of files in a source dir to keep Duplicati busy a few minutes, start the backup and try to restore some files while Duplicati is still backing up. After 20 or 30 seconds the “Database is locked” message appears.
The first post in this thread (job is rescheduled immediately after starting) can be explained by a locked database. In that scenario, the backup cannot be started at all and is rescheduled for the next day.
The question is why the database is locked. It seems that there is only one Duplicati instance. Assuming this is true, the database can only be locked by the same instance.
I guess hibernation could cause the database lock. If the computer goes in hibernation, the computer powers off with the database in a locked state.
After switching on the computer after more than 24 hours (maybe less), the database is still locked. Duplicati doesn’t know that the system is paused for hours/days, but resumes backing up files, thinking there was no hibernation at all.
In the meantime Duplicati detects that it’s time for the next scheduled backup and starts a new one while the first backup is still active (probably without any activity because of the hibernation).
This explains the reschedule without making backups and the “Database is locked” message.
That’s the theory. Chances that is can be reproduced by repeating the steps above: Create a backup, start the backup, put computer in hibernation halfway the backup, wait a day or 2, power on the computer and see what happens.
I Duplicati is set to delay X minutes after startup that would give one enough time to advance the clock 2 days after waking from hibernation - do you think that’s a close enough simlation to test this?
If it doesn’t already, is there a way Duplicati can check if other instances are running? They don’t have to communicate between them - just an “is there another ‘Duplicati’ process running” check might be enough…
Sounds like a good idea to speed things up. However, I’m not sure about:
How long to wait. I guess waiting a few hours is recommendable to ensure that the connection to the backend provider times out before the upload is resumed.
You never know how the backend provider handles a time difference of 2 days. This may cause new, unforseen problems.
If my theory seems to be correct (it’s just a theory to declare @tophee’s symptoms), then it’s the same instance that locks the database. An additional check for running concurrent Duplicati processes is welcome, but it looks like something like this is already implemented. When starting a second instance, Duplicati detects that port 8200 is already in use and will use port 8300 or higher to run additional instances. In a number of scenarios it’s perfectly legitimate to run multiple Duplicati instances (multiple users who have their own Duplicati tray icon and backup jobs).
Thanks for the screenshot - here’s how I interpret it (though I could be wrong):
Duplicati.WindowsService.exe is the actual service that does the backup work
Duplicati.Server.exe Duplicati.WindowsService.exe is what actually starts the Duplicati.WindowsService.exe Duplication.Server.exe service and then pings it periodically to see if it’s still responding, if not then it restarts the service. (Note that this is checking for RESPONSE from the service, not just existence, so it’s really just making sure the service hasn’t hung - see Server Component (Duplicati.Server.exe) for more)
The TWO Duplicati.GUI.TrayIcon.exe client apps seem to be running under different user accounts (based on the width of the anonymized User Name
This makes me think the following POTENTIAL scenarios could be happening:
one or more client GUI is NOT connecting to the service but is using the same duplicati-server.sqlite as the service, so they’re both trying to hit the same DB at the same time
the two client GUIs are using the same duplicati-server.sqlite, so they’re both trying to hit the same DB at the same time
everything is correctly installed and cohabitating peacefully but you’ve got a backup job in each of the client GUIs that share the same --dbpath parameter and just MIGHT happen to overlap in run times
This is just a theory, but (unless you’re using parameters that somehow set custom ports) if you get the GUI on both port 8200 and 8300 then you’ve got either two GUIs or a service and a GUI running independantly.
I’d say a check of your Commandline exports for each backup job to ensure unique dbpath values would be worth doing.
I’m not saying this is what’s going on for you, but it makes me wonder what happens if the tray icon gets put into the “All users” startup folder.
In theory it would start for each user (as it should) and store data in the user-specific AppData folders (as it should). But if the “All users” startup commandline included something like --portable-mode then I fear all users would be pointing to the SAME folder.
It does sound like there are two users running Duplicati (it checks that it only runs once, but that check is based on the folder where the Duplicati-server.sqlite database is located, and easily allows two users to run the backup).
Actually, Duplicati.WindowsService.exe is just a very small monitor program that launches Duplicati.Server.exe and restarts it if it crashes. This is why you see that both processes are running.
The two Duplicati.GUI.TrayIcon.exe exe processes are likely the problem. One of them is running the backup (thus locking the database) and this fails the other.
Multi-user setups are quite tricky because different people want it to work in different ways…
Yes, so I suppose technically you are right. But the other user doesn’t even know what duplicati is. I installed duplicati only under my windows account but apparently it installs itself under all user accounts. In any case, my point is that even though it may be running under a second username, it is not being used there, i.e. that user never opened the gui and certainly has no backup jobs.