Our backups started throwing this error recently and after reviewing a number of posts about this issue, I am unclear what the real fix is?
Can someone please explain what this error is actually stating, and if there is proper solution?
Failed: Unexpected difference in fileset version 1: 6/17/2020 12:30:00 PM (database id: 569), found 142532 entries, but expected 142533
Details: Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 1: 6/17/2020 12:30:00 PM (database id: 569), found 142532 entries, but expected 142533
at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)
at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
Log data:
2020-06-30 12:30:00 -05 - [Warning-Duplicati.Library.Main.Controller-UnsupportedOption]: The supplied option --default-filters is not supported and will be ignored
2020-06-30 12:30:26 -05 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 1: 6/17/2020 12:30:00 PM (database id: 569), found 142532 entries, but expected 142533
at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)
at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.d__2`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.BackupHandler.d__19.MoveNext()
This is the error that is causing backups not to run. (The warning about default-filters can be removed by just deleting that option.)
The easiest way to fix the unexpected difference issue is to delete the offending backup version. Quick instructions:
Click on your backup set in the Web UI, click the Commandline link, then select “delete” from the Command dropdown. Empty the contents of the Commandline Arguments text box. Then scroll to the bottom and pick “version” from the Add Advanced Option dropdown. Enter the version number to delete and click the “Run delete command now” button.
Hi drwtsn32,
Thank you for the assist.
I am stuck at this step “Enter the version number to delete and click the “Run delete command now” button.”
I am unsure what version number I am to enter into this field?
I tried kicking off the backup, but received the following error:
Jul 6, 2020 1:55 PM: Failed while executing “Backup” with id: 1
Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 0: 6/18/2020 12:30:00 PM (database id: 570), found 142580 entries, but expected 142581
at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)
at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass34_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
Do I need to re-run the Fix, for fileset 0? "Unexpected difference in fileset version 0: "
If you get that error when you run a backup, then yes, repeat the DELETE command with version “0”. Hopefully that will be the last issue and you’ll be able to successfully run backups.
I ran it again, but received this return:
The supplied option --default-filters is not supported and will be ignored
Listing remote folder …
No filesets matched the criteria
Return code: 0
I reviewed the logs and found these entries:
“2020-07-06 15:04:38 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Delete has started”,
“2020-07-06 15:04:41 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
“2020-07-06 15:04:53 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (45.56 KB)”,
“2020-07-06 15:04:59 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-PreventingLastFilesetRemoval]: Preventing removal of last fileset, use --allow-full-removal to allow removal …”,
“2020-07-06 15:06:18 -05 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteResults]: No remote filesets were deleted”
],
I then ran the backup again and received this error:
Jul 6, 2020 3:24 PM: Failed while executing “Backup” with id: 1
Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 0: 6/18/2020 12:30:00 PM (database id: 570), found 142580 entries, but expected 142581
at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)
at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass34_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
Oh, how many backup versions total do you have? It sounds like there may be only one left, so Duplicati has safeguards to make sure it isn’t deleted.
Ok, I don’t recall off hand but it’s possible that number doesn’t update after DELETE until the next successful backup. So you have 2 versions, one was deleted, and now Duplicati doesn’t want to delete the last one.
Have you been using Duplicati long or did you just start? Only have 2 versions seems odd to me. What is your retention setting?
I am pretty new to using this app. The client has been using it for a number of years.
This error started within the last month.
I checked the logs and the scheduled backup also failed.
Jul 7, 2020 12:30 PM: Failed while executing “Backup” with id: 1
Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 0: 6/18/2020 12:30:00 PM (database id: 570), found 142580 entries, but expected 142581
at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)
at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass34_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()
at Duplicati.Library.Main.Operation.Common.SingleRunner.d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)
at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
I’m surprised you only keep 2 backup versions. This may make it difficult to recover from this scenario. Where do you store your backups? Is it local or remote? If remote, how large is it?
Worst case - if we need to start over with a fresh backup, is that ok?
And yes you should be able to remove the default-filters option to get rid of that one warning message.
Is this an old backup configuration from 2.0.3.3_beta_2018-04-02? That option’s been gone awhile.
If you like filtering (sometimes it’s helpful), filters are now fine-grained and you have choices below. Filters describes them in general. For specifics, help filter-groups shows personalized info for you.
Typically Duplicati versions are very efficient storage-wise and don’t consume much space (due to deduplication).
For instance on one of my systems I now have 220 versions going back 3 years. The size of the protected data is 30 GB but on the back end all those versions require 80 GB.
You may find that you can retain many versions without it consuming all that much more space.
In any case you may want to start over. The normal “fix” for unexpected difference issue is to delete the offending backup version, but because you only have one version left Duplicati doesn’t want to delete it. If you go ahead and force the deletion, it should remove all files on the back end (AWS). Then when you run your backup again it will have to re-upload everything.
This unexpected difference issue was greatly reduced in the most recent beta (2.0.5.1) but there are still some cases where it happens, unfortunately.
I seem to be stuck on the force delete. I have tried adjusting the retention period and tried running a delete from commandline. I am assuming the default policy is preventing this from completing?
