I have a backup kob scheduled to run at 21:20 every day and a few minutes ago it said:
Next scheduled run:Today at 21:20
But just after 21:20 it just changed to
Next scheduled run:Tomorrow at 21:20
There is nothing at all in the logs. The last log message is 5 days old:
That was a pretty quick run - did it maybe not find any changes so didnât create a any entries anywhere (including the logs)?
If you look at restore options is âtodayâsâ short run listed as one of the versions from which you can restore?
It said 5 versions yesterday and it still says 5 versions, so there was obviously no new version generated. But I checked restore anyway and got this:
Ah - I see now on the original screenshot that the progress bar shows the âPC Drive Dâ backup is still running but the job details say the last SUCCESSFUL run was Tuesday.
So it sounds like the backups may be firing as they should be never finish for some reason - and arenât logging anything about why? The UI almost needs to distinguish between last start time and last end timeâŚ
Does the progress bar count down as expected? Iâm trying to figure out if the job is actually running but not loggingâŚ
Oh, and did you do anything silly like shift from client to service installs (or anything else that might have caused a change in where config or sqlite files would be stored)?
That one is easy: I stopped it. But yesterday, I didnât stop it but the same thing happened as today: it just didnât run.
No, there is no visible progress and there is certainly no network traffic (or disk activity, for that matter) from duplicti.
Yes, I changed to duplicati as a service with some hiccups, but those were resolved and the backup ran fine for at least a day.
OK - unfortunately Iâm out of ideas at the moment, but if I think of something Iâll let you know.
You didnât mention what version of Duplicati you were running, but is it possible youâre running more than one instance (thus causing the locked database)?
Personally, I went from 2.0.2.1 beta client install to a service install then through 3 canary versions and on 2.0.2.9 things totally blew up and I had to uninstall Duplicati all together before a re-install would even let the GUI or service start up.
Through all those updates I think I found (but ultimately destroyed) an scenario of both the service running as well as the client - each with their own server instance. So even though I had it all working with the service at one point, an update (or something I did) split that out into using two different server runs.
Maybe I have an idea what goes wrong:
Duplicati should handle connection interruptions, but after the connection is restored, some cleanup actions could be needed. Partially uploaded files, temporary files and files that are not in sync with the local DB should be detected and cleaned up and fixed. Turning off your computer during a running backup should be allowed, but is not recommended.
Having said that, I guess hiberbation could be a problem in combination with using some particular (or all?) backends. When putting your computer in hibernation, all processes continue after switching on the computer. This may cause conflicts with some or all types of backends.
The connection to the cloud provider probably timed out, so Duplicati has to reconnect and re-authenticate in the middle of an upload. If you shutdown your computer (instead of hibernating), all programs, including Duplicati, will be closed and the backup starts from scratch after powering on your computer.
I guess this could have something to do with the problem that your backup runs forever. After hibernation, connection to the cloud provider is lost while Duplicati is in the middle of an upload and notices that itâs time for a new scheduled backup. The new backup tries to start, but the database is still locked by the previous scheduled run, which will cause the âDatabase is lockedâ message.
This this is the most plausible explanation I can think of for problem that the database is locked and no new backup tasks will start.
Do you have any suggestions on how this scenario could be verified when the issue occurs?
I can reproduce the âDatabase is lockedâ message. Just throw a few hundreds MB of files in a source dir to keep Duplicati busy a few minutes, start the backup and try to restore some files while Duplicati is still backing up. After 20 or 30 seconds the âDatabase is lockedâ message appears.
The first post in this thread (job is rescheduled immediately after starting) can be explained by a locked database. In that scenario, the backup cannot be started at all and is rescheduled for the next day.
The question is why the database is locked. It seems that there is only one Duplicati instance. Assuming this is true, the database can only be locked by the same instance.
I guess hibernation could cause the database lock. If the computer goes in hibernation, the computer powers off with the database in a locked state.
After switching on the computer after more than 24 hours (maybe less), the database is still locked. Duplicati doesnât know that the system is paused for hours/days, but resumes backing up files, thinking there was no hibernation at all.
In the meantime Duplicati detects that itâs time for the next scheduled backup and starts a new one while the first backup is still active (probably without any activity because of the hibernation).
This explains the reschedule without making backups and the âDatabase is lockedâ message.
Thatâs the theory. Chances that is can be reproduced by repeating the steps above: Create a backup, start the backup, put computer in hibernation halfway the backup, wait a day or 2, power on the computer and see what happens.
I Duplicati is set to delay X minutes after startup that would give one enough time to advance the clock 2 days after waking from hibernation - do you think thatâs a close enough simlation to test this?
If it doesnât already, is there a way Duplicati can check if other instances are running? They donât have to communicate between them - just an âis there another âDuplicatiâ process runningâ check might be enoughâŚ
Sounds like a good idea to speed things up. However, Iâm not sure about:
If my theory seems to be correct (itâs just a theory to declare @topheeâs symptoms), then itâs the same instance that locks the database. An additional check for running concurrent Duplicati processes is welcome, but it looks like something like this is already implemented. When starting a second instance, Duplicati detects that port 8200 is already in use and will use port 8300 or higher to run additional instances. In a number of scenarios itâs perfectly legitimate to run multiple Duplicati instances (multiple users who have their own Duplicati tray icon and backup jobs).
I donât think so but Iâm not sure. How do I know if I have two instances? Here is what I have running in terms of duplicati:
So if that means that I have only one instance running, I think @kees-zâs theory sounds like a good explanation.
I can try to reproduce it but first I need to know how I can unlock the databaseâŚ
Thanks for the screenshot - hereâs how I interpret it (though I could be wrong):
Duplicati.WindowsService.exe is the actual service that does the backup work
Duplicati.Server.exe Duplicati.WindowsService.exe is what actually starts the Duplicati.WindowsService.exe Duplication.Server.exe service and then pings it periodically to see if itâs still responding, if not then it restarts the service. (Note that this is checking for RESPONSE from the service, not just existence, so itâs really just making sure the service hasnât hung - see Server Component (Duplicati.Server.exe) for more)
The TWO Duplicati.GUI.TrayIcon.exe client apps seem to be running under different user accounts (based on the width of the anonymized User Name
This makes me think the following POTENTIAL scenarios could be happening:
--dbpath parameter and just MIGHT happen to overlap in run timesThis is just a theory, but (unless youâre using parameters that somehow set custom ports) if you get the GUI on both port 8200 and 8300 then youâve got either two GUIs or a service and a GUI running independantly.
Iâd say a check of your Commandline exports for each backup job to ensure unique dbpath values would be worth doing.
Yes, that is correct and I suppose that is standard behaviour since I only installed duplicati on one (my own) windows account.
The other user is not using duplicati at all (I am doing all the backups from my account (or rather: with the windows service).
Iâm not saying this is whatâs going on for you, but it makes me wonder what happens if the tray icon gets put into the âAll usersâ startup folder.
In theory it would start for each user (as it should) and store data in the user-specific AppData folders (as it should). But if the âAll usersâ startup commandline included something like --portable-mode then I fear all users would be pointing to the SAME folder.
It does sound like there are two users running Duplicati (it checks that it only runs once, but that check is based on the folder where the Duplicati-server.sqlite database is located, and easily allows two users to run the backup).
Actually, Duplicati.WindowsService.exe is just a very small monitor program that launches Duplicati.Server.exe and restarts it if it crashes. This is why you see that both processes are running.
The two Duplicati.GUI.TrayIcon.exe exe processes are likely the problem. One of them is running the backup (thus locking the database) and this fails the other.
Multi-user setups are quite tricky because different people want it to work in different waysâŚ
Except that the other user is not using duplicati at allâŚ
I am confused ⌠the screenshots show that two users are running Duplicati?
Yes, so I suppose technically you are right. But the other user doesnât even know what duplicati is. I installed duplicati only under my windows account but apparently it installs itself under all user accounts. In any case, my point is that even though it may be running under a second username, it is not being used there, i.e. that user never opened the gui and certainly has no backup jobs.