The computer that my Duplicati installation is on sometimes reboots itself while Duplicati is making a backup. This corrupts the database every time. Is there any way to make the PC unable to restart until the backup is over?
Also, I can never seem to repair the backup and move on. I even tried deleting the latest version and resuming from there, it still would not work. This leads to me having to wipe and re-backup the entire drive, which is very time consuming.
Too vague. Please describe this, preferably with steps and any actual error message text.
Windows? Any idea why/when it does that? Second Tuesday Windows Update can do that.
I’m not aware of a generic software way to do that, but maybe specific cause can be helped.
Note this is a Duplicati forum. General Windows support could be more available elsewhere.
What happens?
What happens?
What is supposed to happen, ideally, is any database problems are cleaned up next backup.
Sudden reboot may also cause issue at the destination. What type of storage is destination?
Here is the error message I receive, this happens weekly.
Duplicati.Library.Interface.UserInformationException: The database was attempted repaired, but the repair did not complete. This database may be incomplete and the backup process cannot continue. You may delete the local database and attempt to repair it again.
at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)
at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
I have two backups, an internal and external. Both fail constantly.
The only remedy I’ve found is to delete and restore the database manually, which takes forever and breaks again within the next few days. Using the “delete and restore” option in Duplicati does not work, and results in the same error message.
Thanks. Getting a little closer to the right info, but not quite there.
Duplicati.Library.Interface.UserInformationException: The database was attempted repaired, but the repair did not complete. This database may be incomplete and the backup process cannot continue. You may delete the local database and attempt to repair it again.
RepairInProgress means there was previously a Repair run that failed (probably with messages). Are there any known failed repairs in the job logs or About → Show log → Stored. Do you repair?
Alternatively, auto-cleanup advanced option can do a repair, but it’s a little safer to do it manually.
Regardless, the reason for the failure of repair should be in some messages. If you can’t find any, log-file=<path> will log warnings and errors to disk. If you want more details, add log-file-log-level.
EDIT:
This is a good way to get in a situation that needs repair, but the question is how it didn’t succeed. Possibly you could also just run one by hand, and see if it will fail. Maybe initial failure was similar.
Still working on freeing up space in the drive. I set iDrive and Duplicati to run 12 hours apart from each other, and no crash last night! The machine needs lots of updates but will need to schedule those for later.
All the way back to August, I have this log for my external drive.
Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 0: 8/10/2024 4:03:16 AM (database id: 15), found 1422518 entries, but expected 1422519 at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction) at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass34_0.<VerifyConsistencyAsync>b__0() at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.<RunOnMain>b__0() at Duplicati.Library.Main.Operation.Common.SingleRunner.<DoRunOnMain>d__2`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method) at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter) at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
Followed by
Duplicati.Library.Interface.UserInformationException: The database was attempted repaired, but the repair did not complete. This database may be incomplete and the backup process cannot continue. You may delete the local database and attempt to repair it again.
at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)
at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
And this log for my internal drive.
Duplicati.Library.Interface.UserInformationException: Microsoft Hyper-V VSS Writer not found - cannot backup Hyper-V machines.
at Duplicati.Library.Snapshots.HyperVUtility.<GetAllVMsPathsVSS>d__17.MoveNext()
at Duplicati.Library.Snapshots.HyperVUtility.QueryHyperVGuestsInfo(Boolean bIncludePaths)
at Duplicati.Library.Modules.Builtin.HyperVOptions.RealParseSourcePaths(String[]& paths, String& filter, Dictionary`2 commandlineOptions)
at Duplicati.Library.Modules.Builtin.HyperVOptions.ParseSourcePaths(String[]& paths, String& filter, Dictionary`2 commandlineOptions)
at Duplicati.Library.Main.Controller.SetupCommonOptions(ISetCommonOptions result, String[]& paths, IFilter& filter)
at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
Any ideas?
EDIT/NOTE
I am using Duplicati to backup my machine to both an internal and external drive. They are scheduled to backup one minute after the other.
As the message shows later, there is a difference of opinion on the file count of the version.
Version 0 is always the latest, but you can also check that against the Restore menu listing.
would be the thing to do immediately after the message. If not the latest, delete what’s bad.
Sometimes multiple versions get the count error, but I think the current 2.0.8.1 Beta lists all.
Have you ever tried a Database Recreate? If all goes well, that should be pretty fast.
You can tell when things are not going so well if the progress bar slows beyond 70%.
The “Unexpected difference in fileset” is an open issue. I don’t know if it’s understood.
It’s detected before or after any backup, I think, and probably also by Verify files.
I don’t think Repair fixes it, but maybe it notices and so says “repair did not complete”.
Possibly you could try a manual Repair to see if you can get “Unexpected difference”.
If so (and if it’s a reasonable damage list), delete the bad versions in a Commandline.
You could also try database recreate, but save current DB in case recreate goes bad.
Going about different fixes now. While trying to free up disk space, I found a 25 gig etilqs (sqlite) file in my temp folder. It hasn’t been modified in over a year. I’m assuming it’s an old duplicati database. The file name is “etilqs_QLI7V5vSEFeDJf9”. Is it safe to assume I can get rid of this?
Was looking for extra space because I ran “repair” on my internal drive and was met with “Unexpected update count: 0”. A Google search led me to believe this was caused by low disk space, but I have 30 gigs free on my C:\ and am still getting this error when I try and repair.
2024-09-25 15:21:25 -04 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Repair has failed with error: Unexpected update count: 0
Exception: Unexpected update count: 0
Any ideas? It’s like the more I try and fix things the more errors I run into.
What did you do before that? We were also talking about Recreate. List all steps you recall.
The blocklist hash area relates to blocklists, and there was a blocklist bug fixed in a Canary.
Although I don’t know if it might help whatever you did, how would you feel about a Canary? Generally these are testing releases with the latest fixes plus the latest bugs that crept in…
EDIT:
Your job logs will show some history, but logs get erased if you Recreate. You see new work.
Unfortunately I’m taking over trying to fix this for someone else, this isn’t even my own server. Right now the goal is to be able to easily repair the database if it becomes corrupt. As long as we can reliably have access to the backup and backup history we’re satisfied.
I’ve repaired and recreated numerous times and keep ending up with multiple errors. I’m not sure what the owner of this server did initially once the backup hadn’t worked correctly.
Currently recreating the database for my external drive. I manually renamed the original DB, hopefully it works nicely when it’s done.
The internal drive gave me the error while attempting to make a backup.
Found 2 file(s) with missing blocklist hashes
I read a post on this forum that said I could probably fix this by repairing the DB, going to try once the external is done recreating.
EDIT: This post seems to be exactly what I’m experiencing with my internal drive.
EDIT 2:
My external drive is stuck on “recreating database”. The drive has no activity according to task manager and the progress bar has not moved since earlier this morning. The last update in the log was
Cannot find any MS SQL Server instance. MS SQL Server is probably not installed.
What sort of log is this? Was it running before? If so, what was going on before such messages?
About --> Show log --> System info might have something in lastPgEvent for last status.
Roughly where is it, as it goes left to right? A good log level to match to the bar would be verbose, although Information can sort of do that, with less explanation. If need be, log detail can be raised.
Symptom matches, but it was fixed years ago. Problem is that you can’t get to Repair to run it.
These are the errors I have under “Information” in Live Logs.
Oct 1, 2024 12:05 AM: The operation Backup has failed with error: The database was attempted repaired, but the repair did not complete. This database may be incomplete and the backup process cannot continue. You may delete the local database and attempt to repair it again.
{"ClassName":"Duplicati.Library.Interface.UserInformationException","Message":"The database was attempted repaired, but the repair did not complete. This database may be incomplete and the backup process cannot continue. You may delete the local database and attempt to repair it again.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)\r\n at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)\r\n at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)\r\n at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nMoveNext\nDuplicati.Library.Main, Version=2.0.8.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__20\nVoid MoveNext()","HResult":-2146233088,"Source":"Duplicati.Library.Main","WatsonBuckets":null}
And
Oct 1, 2024 12:06 AM: UsageReporter failed
{"ClassName":"System.Net.WebException","Message":"The remote server returned an error: (504) Gateway Timeout.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()\r\n at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()\r\n at Duplicati.Library.UsageReporter.ReportSetUploader.<>c.<<Run>b__3_0>d.MoveNext()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nGetResponseOrStream\nDuplicati.Library.Utility, Version=2.0.8.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper\nSystem.Object GetResponseOrStream()","HResult":-2146233079,"Source":"Duplicati.Library.Utility","WatsonBuckets":null}
Also, my database (the new one I started making last week that was stuck at 75%) is currently marked as “in-progress”. Duplicati currently isn’t doing anything. It seems there’s a hangup while making the database that never ends up being resolved.
How can I exclude the Database from backups? I see on multiple forum posts that this is recommended. Also, we have Duplicati set to backup multiple active Hyper-V VM’s, could it be that since these are constantly changing, Duplicati is picking up on it as a “corrupt” or “changed” file?
As always, what is the history leading up to this? Any repair? Check logs. Any problem in it?
Something seems to be going wrong, but seeing only a message it didn’t finish adds no info.
Probably unrelated and not to worry about unless it repeats. Maybe server was down awhile.
Duplicati can send anonymous reports containing information about how you use Duplicati. These reports can be viewed at https://usage-reporter.duplicati.com/.
The one below from five days ago?
Since you have both an internal and external destination, please be clear which you describe.
How do you have Duplicati stuck at 75% on a recreate, while it’s also trying to start a backup?
For any such problem, a log gives more detail than progress bar length (centimeters is easier, should measuring be essential, as the math is easier than guessing at eighths, etc. of inches).
Verbose level would be a good starter for Recreate issues, as it’s quite descriptive of process.
says that if you’re really at 75%, it’s just barely starting dblock reads, but log will say it clearly.
Additional mystery, though, is why it stops, especially since the attached drive should run OK. Internet access to remote storage has more things that can go wrong, causing possible hang.
There’s some chance that About → Show log → Live → Verbose would show the last history, however it’s intended to be set up beforehand, to watch live. Going back in time may not work.
Clicking an item in the file picker will add that item and all child items to the source selection list. This is indicated with a green check mark. Clicking it a second time changes the check mark to a red cross. This excludes that item an all child items from the backup.
is the job database, which has (as you mention) been worrisome, but that’s mostly for backup.
seems to say it should require VSS (I don’t have Hyper-V VMs). If so, VSS freezes changes. Backing up files that change during backup runs fine, but you might prefer more consistency. Corruption in specific application files isn’t something Duplicati looks for. They’re merely files.
As before, input from the developer or other experts is welcome. Meanwhile, any better info?
