Yet another “Unexpected difference in fileset version” error

#1

Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 0: 1/9/2019 1:13:01 AM (database id: 1), found 161 entries, but expected 435 at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction) at Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.b__0() at Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0() at Duplicati.Library.Main.Operation.Common.SingleRunner.d__2`1.MoveNext()

Tried removing the custom retention, no change. Tried deleting versions 0 and then 1, no change. Have repaired the database, then recreated the database. No change. The repair operation did have some stuff to say:

Duplicati.CommandLine.exe repair “googledrive://Duplicati backups?xxx” --passphrase=“xxx” --dbpath=“C:\ProgramData\Duplicati\Data\backupmystuffnewdb.sqlite” --verbose
Downloading file (39.67 KB) …
Downloading file (17.98 KB) …
[ … ]
Remote file referenced as duplicati-bc96f7cc528014a41a9fb1ff177103cfb.dblock.zip.aes by duplicati-ia130a5f0e9ba42d6a49b3b206dc898b2.dindex.zip.aes, but not found in list, registering a missing remote file
Downloading file (27.73 KB) …
Remote file referenced as duplicati-b362f5606fdd140e88e429ed252907da6.dblock.zip.aes by duplicati-ic249d61a1f78448abe0af70843ad1e12.dindex.zip.aes, but not found in list, registering a missing remote file
[ several more of these ]
Remote file referenced as duplicati-ba33e453462aa4ab7a53a87c0a5be3a6c.dblock.zip.aes by duplicati-i2958921989ec45a596fc8d225b50fcfc.dindex.zip.aes, but not found in list, registering a missing remote file
Found 24 missing volumes; attempting to replace blocks from existing volumes
Found 21 missing volumes; attempting to replace blocks from existing volumes

A second repair operation had no interesting output.

Do we have any wisdom on how to recover from this error? Thanks in advance.

#2

Some more data, in the hope that it will be useful:

In the Restore part of the WebUI, it says:

Back up my stuff
90.68 GB / 18 Versions

That’s the name of the broken backup. When I select that backup to restore from, there is only one version listed:

0: Jan 9, 2019 1:13 AM

The inconsistency in the number of versions caught my eye.

#3

Yet more data, if this helps at all. Getting a 500 return code, I presume from the Google Drive API?

C:\Program Files\Duplicati 2>Duplicati.CommandLine.exe test “googledrive://Duplicati backups?authid=xxx” all --dbpath=“C:\ProgramData\Duplicati\Data\backupmystuffnewdb.sqlite” --passphrase=“hunter2”

ErrorID: FilesetDifferences
Unexpected difference in fileset version 0: 1/9/2019 1:13:01 AM (database id: 1), found 161 entries, but expected 435

C:\Program Files\Duplicati 2>Duplicati.CommandLine.exe delete “googledrive://Duplicati backups?authid=xxx” --dbpath=“C:\ProgramData\Duplicati\Data\backupmystuffnewdb.sqlite” --passphrase=“hunter2” --version=1 --dry-run
Listing remote folder …
Listing remote folder …
Listing remote folder …
Listing remote folder …
Listing remote folder …

System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at Duplicati.Library.Main.BackendManager.List()
at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)
at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)
at Duplicati.Library.Main.Operation.DeleteHandler.Run()
at Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action1 method) at Duplicati.Library.Main.Controller.Delete() at Duplicati.CommandLine.Commands.Delete(TextWriter outwriter, Action1 setup, List1 args, Dictionary2 options, IFilter filter)
at Duplicati.CommandLine.Program.ParseCommandLine(TextWriter outwriter, Action1 setup, Boolean& verboseErrors, String[] args) at Duplicati.CommandLine.Program.RunCommandLine(TextWriter outwriter, TextWriter errwriter, Action1 setup, String args)

#4

I would really appreciate if someone could help out with this issue.

I have been battling the “Unexpected difference in fileset” issue since December 2018 when I unfortunately updated Duplicati 2.0.4.5_beta. Before the update it worked great for over a year. One of the early threads can be found here: Backups not running/error out after update to 2.0.4.5_beta_2018-11-28:

So far, my only course of action has been to recreate the backup(s) and start new. Repair, Recreate database have never worked. At this point, I have had to do this 6 times (I have 2 seperate computers running and 3 total backup routines). Just this week, 2 of my routines started doing giving me the error again and I am giving up until there is some type of update. For me, it takes about a month or so for the error to pop up on a new backup routine.

#5

What works for me is to just delete the offending backup version. It is quick and pretty painless.

I haven’t been keeping close tabs on this issue to know if it has been fixed in later canary versions or not. If so I’ll definitely be eager to upgrade to the next beta when it’s released!

#6

Thanks for the advice.
I decided to give this a try. At first the error was Fileset version 6. Deleted, then it said fileset version 3, then 2, then 1, then 0. I deleted them all and it appears to now be running a backup!

Is there any way to just dump everything and restart the backup without having to go through each fileset and/or deleting an re-creating the backup routine?

Does anyone know if this issue been identified and fixed in any of the new versions? I have gotten to the end of my rope having to re-create my backups ever month or so.

#7

Strange, I have never had to delete more than one version to (temporarily) fix this issue. I usually just delete the one and then backups are good again.

#8

Well thank again for your help. The other routine which was doing the same thing had 11 versions. To fix, I had to delete version 1 and version 0. Either way, its much easier than completely rebuilding the backup.
Thanks again!