Fatal error: Detected non-empty blocksets with no associated blocks

#21

What --log-file-log-level did you use? I suggest verbose.

If verbose is too much, try Information

What would be helpful (but it’s a lot of steps) is

  1. Starting with a working backup job
  2. Turn on --log-file-log-level (as you have)
  3. experience non-empty blocksets error (well, I hope you don’t, I hope no one does, but if you do)
  4. perform sql query in database browser
  5. Find 5 of 10 files “around” where the failing file was
  6. go back to logfile and search for those files. This is likely from previous runs, not the current that received the error.
  7. hopefully something in the logfile will be revealing
Fatal error System.Exception: Detected non-empty blocksets with no associated blocks!
#22

@thomasp your PathTooLongException appears to be a known bug with --usn-policy. See this post.

I updated my reply #16 with an edit to this effect.

#23

As a followup, I ended up adding filters to prevent Duplicati from trying to backup both the temp directory as well as the directory where Duplicati’s database was being kept, along with a few other directories containing some Windows files that were constantly changing and didn’t need to be backed up. Then I recreated the database. It’s been running for somewhere around a week and a half without a problem. Seeing as how this error previously kept being generated within a day or two of recreating the database, and that happened several times, I feel pretty confident that these exclude filters are what is fixing it for me and should point the developers toward fixing it in the code.

#24

Interestingly, as soon as I turned on the most verbose log level for a job where this had happened, this error has stopped happening to me. So, either I got lucky so far, or an update has fixed it?

1 Like
#25

Are you using OneDrive backend?

I have two backups to two different OneDrive accounts running for more than a year. 15 days ago, one of them failed with “Detected non-empty blocksets” error. Repair database don’t change anything, and recreate database (it took one week) turns the error to “You have attempted to change the block-size on an existing backup”.
I’ve deleted the backup (bye bye 6 months of backup history) and recreated the task and now I’m getting “Object reference not set to an instance of an object” in different files everytime I try to run the task.

The other backup task is working right all the time so I suspect the OneDrive account. I had other bad experiences in the past with OneDrive. Microsoft guys changes something or something in some servers gets unstable and stops working (and long after they fix it and works again)

#26

I’m using local disk backup, and just today had this problem once again. It just randomly hits you. After running manual repair, backups work again.

#27

It seems to me that Duplicati isn’t very robust to failures of the program. While I was backing up for the first time (testing this out) my computer’s battery died and so the whole machine shut off. Also, after that I had to force quit Duplicati because it wouldn’t pause. Backup software like this should be able to recover from situations like that, but I’m not sure Duplicati is robust enough on those fronts yet. After those things happened, Duplicati gave me errors about a lot of things, but kept going and seemed to maybe be working? That is until I go to restore and it gives me this “non-empty blocksets” error. After attempting to restore and getting that error, I no longer see any directories when I go to restore. This isn’t very encouraging : /

Ugh, and if you rename the directory where the backup is stored, it won’t remove the backup from your home screen! It really seems like a lot more errors need to be gracefully handled. Hmm, after restarting the duplicati service, it seems to have gotten the picture and not display the backup.

1 Like
#28

Without much technical knowledge, I may add the following experience, which occured by chance:
When Duplicati showed this error “Detected non-empty blocksets with no associated blocks”, and repair and other suggestions I’ve found here didn’t make the error go away, I finally choose to recreate the back-up. To do that, I have created a new backup from the config file of the old error-prone backup, renamed it, but - forgot to change the destination folder!
I ran it, and the above mentioned error did not reappear. I hope this new backup does what it is expected to do.
Running Duplicati 2 as a server on Win10, backup to external HD. Hope this helps.

My question: Can I trust this backup, since it still gives me a few strange errors?

2019-04-30 05:28:48 +02 - [Warning-Duplicati.Library.Main.Operation.BackupHandler-SnapshotFailed]: Fehler beim Erstellen des Snapshots: System.UnauthorizedAccessException: Es wurde versucht, einen nicht autorisierten Vorgang auszuführen.
bei Alphaleonis.Win32.Vss.VssBackupComponents…ctor()
bei Alphaleonis.Win32.Vss.VssImplementation.CreateVssBackupComponents()
bei Duplicati.Library.Snapshots.WindowsSnapshot…ctor(IEnumerable1 sources, IDictionary2 options)
bei Duplicati.Library.Snapshots.SnapshotUtility.CreateWindowsSnapshot(IEnumerable1 folders, Dictionary2 options)
bei Duplicati.Library.Main.Operation.BackupHandler.GetSnapshot(String sources, Options options),
2019-04-30 05:28:48 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\Documents and Settings\Hannah,
2019-04-30 05:28:48 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\Documents and Settings\Hannah,
2019-04-30 05:28:48 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\Documents and Settings\DefaultAppPool,
2019-04-30 05:28:48 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\Documents and Settings\DefaultAppPool, … ]
Errors:
[ 2019-04-30 06:48:30 +02 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-CheckingErrorsForIssue1400]: Checking errors, related to #1400. Unexpected result count: 0, expected 1, hash: 6y9axTuPuScijYqpJYoSIrWjr4oCT8anZyaWz45/PCw=, size: 102400, blocksetid: 2323039, ix: 15, fullhash: +9taBJbTJQPh6PnJfFf8LwBNUwtcmipp+dAh1VRZXLM=, fullsize: 1770112,
2019-04-30 06:48:30 +02 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-FoundIssue1400Error]: Found block with ID 7194036 and hash 6y9axTuPuScijYqpJYoSIrWjr4oCT8anZyaWz45/PCw= and size 28136,
2019-04-30 07:33:32 +02 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-CheckingErrorsForIssue1400]: Checking errors, related to #1400. Unexpected result count: 0, expected 1, hash: RchjnSGyxBpoBYVUF+eTOJs01I7NwR+px7xJ6/OHNsc=, size: 102400, blocksetid: 2323154, ix: 4, fullhash: et74+L9UlZs0anjdoQfPQdLus4M60e7CcV1TdNlF44Y=, fullsize: 2504664,
2019-04-30 07:33:32 +02 - [Error-Duplicati.Library.Main.Database.LocalBackupDatabase-FoundIssue1400Error]: Found block with ID 7199469 and hash RchjnSGyxBpoBYVUF+eTOJs01I7NwR+px7xJ6/OHNsc= and size 21832 ]

My question would be: Do these errors compromise the backup?

#29

I just received this error last night. The backup client is running Linux. The backup server is a nextcloud instance using webdav on a Linux system. I do not have snapshots turned on for this backup system.

The result of the first query is

sqlite> SELECT * FROM Blockset WHERE Length > 0 AND ID NOT IN (SELECT BlocksetId FROM BlocksetEntry);
ID|Length|FullHash
2359268|7804|gR5EPkHAuZLY+iQuMY0YX6tW36r3r/SAYq+K/Pr3+Fg=

The result of checking for that in the Files table is no result

sqlite> SELECT * FROM File WHERE BlocksetID = 2359268;

I’ve copied the database off to the side so that I can run any other queries on it that will help to debug this issue.

I’ve started a database repair and will see if this works. Otherwise it’ll be a few days to create a new backup.

#30

I had this error a few months ago. I remember retrying the same backup over and over with fresh installs of duplicati. It keeps failing. Trying to repair? Has to download and unzip everything. Fell back to previous beta. Can’t believe the devs haven’t fixed yet…

#31

The key to allow a fix is a good test case that reproduces it, and (from skimming here), I think this is still a random thing. If anybody on the thread has found a good reliable way to get it, or clue as to what causes it, please point. I’m just skimming the rather long thread, but not seeing much that would help the developers.

Other things that may help are a database bug report as mentioned earlier, or logs at as detailed a level as one can stand, also mentioned earlier. Since it’s been many posts back, that means Creating a bug report and/or –log-file with –log-file-log-level=Profiling plus --profile-all-database-queries (it will create a huge log).

Maybe easier is to let development do the heavy debugging, provided someone who hits it on a somewhat regular basis can simplify it into steps, so development can create it for debug. Filing an issue on it is best.