I recently had to stop a backup partway through due to the target server losing power and needing to shut it down before the UPS died and I made sure that after it was stopped that it showed that it had the backup and it said that it had 3 versions and was using the correct amount of space. Now after that and having restarted the docker container duplicati is running in it says the backup has never been ran, but when I go to recover data from the backup, it sees the versions and lets me recover the backed up files. I have tried first a database repair and after that didn’t work I then tried to have it recreate the database and neither have worked. Is there any way to fix this since I would really not need to redo the backup since it was many TB’s in and would take days to backup again.
If you can restore the versions, you should be fine. The display underneath the backup is saved in a different database (the server database), and if the last backup is aborted it might display 0 bytes / 0 versions in that field. If you run another backup, it should be correct again. The most important thing is that the restore page lists all the versions.
Does it look like this screenshot? Then you should be okay to just continue using the backup.
It says the Last successful backup was never. I clicked run again, but it was starting from over which would take days to recreate so I was hoping I didn’t have to do that.
That might be because you did a database recreate. Can you see the versions on the restore page? Test that some files will restore.
How did you determine this? It would still need to scan all the files for changes after each new backup, but it will only upload changes, so it should be much faster than a full restart.
That might be because you did a database recreate. Can you see the versions on the restore page? Test that some files will restore
It said that the last successful backup was never after I did a database repair and before I recreated the database. I was able to restore a file and checked and made sure the file was correct and it was.
How did you determine this? It would still need to scan all the files for changes after each new backup, but it will only upload changes, so it should be much faster than a full restart.
I determined that by running the backup and it started from the beginning.
The progress bar will always start from the beginning. Duplicati does incremental backups, so it will go through all the files and detect if there are any changes (based on timestamp usually). This step must run for every backup, and there is no way to continue an interrupted file scan.
If differences are detected, only the changed data is uploaded to the backup destination. This will take into account any interrupted backups. If there are no changes since the last version, it should run faster because it doesn’t need to read the file contents. Because you can restore fine, your backup should be good. However, since your last backup was interrupted you should run another to ensure that all the data is backed up.
It seems that since I did a database recreate it just ignored the stuff previously backed up, but I remembered that I backed up all the duplicate config/database files from before I ran the database recreate and after doing that and resuming it is now resuming.
Thanks for the help.
I don’t know where this landed, but the early not-well-described repair and recreate failures are worrying because if the destination cannot be turned into a database, you’re at risk if the Duplicati system breaks. Duplicati.CommandLine.RecoveryTool.exe is a less pleasant alternative if you have a corrupted backup.
Restoring files if your Duplicati installation is lost generally needs to create a partial temporary database.
I personally prefer not to just click the button labelled Recreate (delete and repair), as recreate of new database may fail, and I prefer having old one to fall back on. Is such a fallback what you did here?
It’s a good idea to periodically test that database rebuild works, as disaster recovery may have to use it.
This whole topic started off somewhat mysteriously. Were all the shutdowns and startups clean, or was power lost early? I’m no Docker expert, but generally I think it’s supposed to come up like it went down.
Are there other backup configured in this Duplicati? I don’t know how this one managed to say “Never”.
I can cause a “Never” by deleting LastBackupFinished from Duplicati-server.sqlite, but how yours got that way is unclear. I hope you’re storing configuration on the host, and not inside the Docker container.
Is such a fallback what you did here?
Yes, I made a file copy of the entire config folder for Duplicati & when I realized that my database recreate screwed it up, I restored that copy and then I was able to correctly resume the backup.
Were all the shutdowns and startups clean, or was power lost early?
This started when I was doing the initial backup for a large dataset and my servers got unplugged and I had to stop the backup before the UPS ran out so I stopped the backup inside duplicati then once it had stopped and showed that it had a version and that the backup was the right size so I shut the servers both down cleanly and when I was able to turn them back on it then said this backup had never ran even though it hadn’t said that before the off & on.
Are there other backup configured in this Duplicati?
I think like 10 others that have been working fine for what I think has been at least a year now.
I hope you’re storing configuration on the host, and not inside the Docker container.
I am of course storing it on the host, I’m not trying to live that dangerously.
So in this one Duplicati, the others came up fine, and the one mentioned came up “Never”? Odd.
Clearing a specific value in a long-running job in a single Duplicati (without other loss) is strange.
Well, at least there’s some history given in case someone decides they want to try to repro issue.
All I can advise is to make sure that the affected one is now fully healthy. Backup doesn’t prove a
recreate-database-from-backup-files-on-destination will work. If you recall details from the original
repair and recreate failures, that might be a clue. Or try them again, carefully. Testing adds safety.
I missed that part before, and it might be significant as it makes “Never” closer than a long job history.
Although you did verify things, I suppose it’s possible that the database didn’t record the backup right.
I don’t know the code, but sometimes data that was once there is lost after code doesn’t do a commit.
Database transaction will be rolled back on next start. It’s a database feature that can cause surprise.
Having an established run history that disappeared would be even harder than not recording an initial.
So in this one Duplicati, the others came up fine, and the one mentioned came up “Never”?
I missed that part before, and it might be significant as it makes “Never” closer than a long job history.
I was able to stop it cleanly and before the server went off I checked and it was displaying correctly, only after the server turned off then back on did it say never.
All I can advise is to make sure that the affected one is now fully healthy.
I plan on checking after it finishes the first full backup in a couple days.
That’s what happens if the data somehow never got its commit.
Commit (data management) (Wikipedia)
In computer science and data management, a commit is the making of a set of tentative changes permanent
Just a theory, lacking any other…
EDIT:
Test of theory is not confirming it. Created a new backup job, Run, ask Stop now, check home screen:
Database has no associated -journal file, so it’s not in a transaction. I can also see the needed data.
Stop Duplicati and restart it to see if it gets different. Looks the same, including preserved Notification.
calls it a partial backup because I did a Stop rather than letting it finish. I waited until uploading began.
You (or anybody) is certainly welcome to see if they can come up with reproducible steps to get issue.
Having no need for Docker would be best. Basically try to use a setup that is very easy for developers.
If you get steps, put them in a new GitHub Issue so that the developers can look into it at some time…