Reproducible: Unexpected difference in fileset version X: found Y entries, but expected Z

bug

#1

I’ve seen a few posts about this error and how to solve it, but it’s never mentioned what’s the cause of it…
I’ve had this error on a few of my backup jobs, and it seemed the reason was mostly because the job was interrupted for some reason like a network outage or system reboot.

Today I managed to reproduce the error consistently, and I hope this can help debug the root of the problem. Here are the steps I took:

  1. Create a backup job with the destination set to SSH/SFTP.
  2. Start the job, and let it upload a couple of volumes.
  3. Attempt to stop it by requesting to “Stop after upload”. It is a known bug that this will not actually stop the job, and it will go on.
  4. Now stop the job by selecting the option “Stop now”.
  5. Attempt to start the job again. The error Unexpected difference in fileset version X: found Y entries, but expected Z will be generated.

Here are the settings I use:

Running Duplicati-2.0.4.12_canary_2019-01-16

General Settings:

--allow-sleep=true
--usn-policy=Auto
--tempdir=E:\Temp\SYSTEM\Duplicati
--retry-delay=20s
--number-of-retries=10
--dont-compress-restore-paths=true
--concurrency-block-hashers=6
--concurrency-max-threads=8
--concurrency-compressors=3

Job-specific settings:

--ssh-fingerprint=ssh-rsa 2048 {xxx}
Remote volume size: 10MB
Retention: Keep all backups

Error from the internal log:

Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 1: 2019-01-30 16:19:11 (database id: 1), found 76 entries, but expected 417 at 
Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction) at 
Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.<VerifyConsistencyAsync>b__0() at 
Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.<RunOnMain>b__0() at 
Duplicati.Library.Main.Operation.Common.SingleRunner.<DoRunOnMain>d__2`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at 
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at 
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at 
Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext() --- End of stack trace from previous location where exception was thrown --- at 
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at 
CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at 
Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result) at 
Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method) at 
Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter) at 
Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

Error Unexpected difference in fileset version
#2

I stumbled upon it on a communications glitch. See “Unexpected difference in fileset” errors [2.0.3.9], and what I had to say. Did you happen to test it to a local file? If “Stop” can get it there, it’s an easier test to run. Still, your test is easier than where I left off, which involved cutting off the communications at the server…

Thanks for helping track it down. Getting a good reproducible test case is (IMO) key to understanding and fix. Yours actually looks good enough to file as an issue where the development team might start working. Would you be able to file? In addition to attracting developers, issues get tracked better than posts here…


Problems with the dbase backup
#3

I’m happy I could be of help :smiley:
I’ve created the issue here. Thank you for suggesting that!