The only frequently-successful repair that I know of is to delete the fileset version, which is probably what @haus was suggesting (unless somehow it now requires deleting all the versions). What’s your history with the error? That might hold you for awhile if it’s rare. It’s too frequent for the two who have just posted.
Is there any way to fix an “unexpected difference in fileset” error other than deleting? is @drwtsn32 post although the original poster was also trying to avoid doing this, but troubleshooting there didn’t get very far.
The easiest way to delete a fileset is probably using the job’s GUI Commandline and adjusting the screen.
The BACKUP command syntax gets turned into The DELETE command by deleting the source files from Commandline arguments and adding --version=4. Change Command at top to delete, then Run at bottom.
Getting the error to stay away has been a problem for some people, but I hope we can use their results to try to figure out why it only seems to affect certain backups. There has definitely been a network-problem cause identified by me here, reproduced by @warwickmm here, leading to possible code fixes at below:
Ensure that failed put operations throw exception #3697
The problem is that this is not the only cause. People have seen it on backups to local disk. What’s yours?
Manual compact ran, now unexpected difference in fileset is @haus finding that compact without enough temporary file space can cause this. There have also been theories involving things like retention deletions.
I’m open to all kinds of theories, but rely on people who can see it to try to guess what encourages it and to consider adding logging, consider posting logs, and consider even posting database bug reports which are sanitized versions of the database, e.g. with source pathnames and other data removed to protect privacy. Posting big files here is not possible, so you’d post a link to a cloud storage share, Firefox Send, or similar.
So deleting versions stopped helping even temporarily? I believe deletion can also cause auto-compact, so using --no-auto-compact on the delete command might see whether delete itself moves the problem into a different fileset, or if it’s one of the follow-on operations (such as possible) automatic compact that did that. Ultimately this is the kind of thing that database dumps and the right expert (maybe not me?) can help with.
The “unusual” is sometimes the run before, at least it is in the case where network errors on a put upload causes Duplicati to bail. It does this same sanity test twice, once before a backup, and again after (but if it gets an exception of some sort, that might prevent the post-backup sanity test – so it hits that on next run).
This is actually a nice simple backup in a way. Did it say it found 5 entries, but expected 6? If so, then the display might be going by the left number, which I think is more realistic in terms of what can be restored.
We’re not likely to hear from the original designer, however this is one of the self-checks Duplicati does to make sure its various database records are in order. Programs are typically written to rely on things being the way they should be. If things look wrong, sometimes it’s better to stop than to spread the mess further.
Technical details of this internal check are found here. Count from those SQL queries should be the same. With such a small list, you could even humanly compare the file lists with an SQLite browser such as this.
For those of you who are seeing the error too much, I’d suggest looking in logs for compact operations just before it. Here is an example of how it looks in job logs. You can also see if using –no-auto-compact helps, change Backup retention to Keep all backups, and (if network backup) set –number-of-retries higher.
Fileset 0 is the newest, so damage to an older fileset such as 4 (with 16 versions total) seems like it might be some sort of non-backup maintenance work such as a compact coming through, doing wider changes.
Though there are reports of this happening on systems that are always on, it’d still be a good idea to see if that happens to you, or if things like sleep or restart tend to encourage this problem to show up sometime.