Backup - Detected non-empty blocksets with no associated blocks!

ArdieX · January 21, 2019, 12:23am

My first backup just finished running (approx. 250GB to an S3 (Minio) destination) and it took about two days, there were some warning during the backup of temp/locked files and I have de-selected those from the backup source and tried to run the backup again and receive the error:

Error while running BACKUPNAME
Detected non-empty blocksets with no associated blocks!

I believe that over the two days there may have been a disruption to the network connection as well but the backup seemed to continue.
I tried running a Repair operation on the database and it completes successfully and says everything is in sync.

I saw another recent post on the forums but that user ended up re-running their entire backup. which since I just uploaded 250GB on my internet connection I don’t want to have to start over.

I am running: Duplicati - 2.0.4.12_canary_2019-01-16 on Windows 10 Pro 1809
I have attached the initial backup log.

Can anyone provide some guidance in tracking down the source of this error?

Thanks.backup.zip (2.5 KB)

ArdieX · January 21, 2019, 1:39am

I did a bit more reading on some of the other posts with this error and I have found one commanlity, attempting to back up the C:\Users\username\ profile folder which includes AppData which obviously will contain running/locked files.

Specifically I have already seen mention of the Chrome folder errors as a precursor to this issue that I also have in the above attached log:

"2019-01-18 18:04:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\\Users\\ArdieX\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Cache\\f_00b5bc", "2019-01-18 18:04:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\\Users\\ArdieX\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Cache\\f_00b5bd", "2019-01-18 18:04:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\\Users\\ArdieX\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Cache\\f_00b5be", "2019-01-18 18:04:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\\Users\\ArdieX\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\Cache\\f_00b5bf",

There was an SQL query provided in one of the other threads as well, when I execute it on my database I see two results (hashes modified).

|“212008”|“5433”|“En8JBIKjoSl4juwUV3Xcyw1XXXXXXXXXXXXXXXXXXXX=”|
|“212012”|“18618”|“pLuz0EW0bGEuF1c4edoP9ZPXXXXXXXXXXXXXXXXXXXX=”|

From here I’m not sure what to do or check or if there is even any feasible solution other than re-running my entire backup.

Thanks.

johnvk · January 22, 2019, 3:05am

Good Work! Thanks!

I’m adding your post to the list in this post

Did you do the next step–taking these IDs you and looking them up in the File table in its BlocksetId column? I see that like others, your IDs are close together numerically. Look at neighboring IDs. It would be nice to see if they are indeed chrome temp files.

I am guessing from your results you did not run prived, and also did not use --snapshot-policy=on. Is that correct?

On your repair, did you run it via the cmd line or the GUI? In either case can you check the Duplicati working dir for its .sqlite files? When I ran the cmd line, the repair worked, and it created a new .sqlite file, but the job was still using the old db file in that dir. The db file is listed in the job settings, if you expand out the drop-down under the job name, under “Advanced” there’s “Database…”. The folder on windows is C:\users\username\appdata\local\duplicati\ when running trayIcon, and C:\Windows\System32\config\systemprofile\AppData\Local\Duplicati when running as a service.

ArdieX · January 22, 2019, 4:14am

I didn’t see that at the time, I ran:

SELECT * FROM File WHERE BlocksetID BETWEEN 211995 AND 212020

And returned the following:

|“151120”|“C:\Users\ArdieX\AppData\Local\Temp\dup-336dbedc-e1a2-4d63-8881-f44efadc2ebf”|“212004”|“125866”|
|“151121”|“C:\Users\ArdieX\AppData\Local\Temp\dup-45abce82-43e4-402a-a05f-5b34363e1f10”|“212006”|“125867”|
|“151122”|“C:\Users\ArdieX\AppData\Local\Temp\dup-d90be97e-512f-462e-8400-c0943b2ec14a”|“212010”|“125869”|
|“151124”|“C:\Users\ArdieX\AppData\Local\PrivateInternetAccess\User Data\Local State”|“212015”|“125872”|

Im not sure what you mean by prived, I just launch Duplicati from the systray, however my user account is a local admin if thats what you mean. It was installed and run as the same and only user on this system.

I have not added the --snapshot-policy option.

The repair was run from the GUI and it did not make any new files, I assume this is because it reported the database had no errors so there was no new .sqlite DB created.

i have not tried the DB “Recreate” option as I am not sure what effect it has yet and if I will need to re-run my entire backup.

Thanks for your help.

johnvk · January 29, 2019, 12:19am

Just to clarify (I know it’s been a few days), did you say you got the “non-empty blocksets” error, rgwb ran the database repair which said “no errors”. And this is all on the same job?

Cuz that would mean there are errors in the database (eg, this error) that “repair” does not find and/or fix.

And if so, running a backup after the database repair, did the error go away or still there?

As for privs, from your description, you are not “running prived”. And --snapshot-policy=on requires privs so that is consistent. You’re not using it.

As you say you are including your whole user folder including AppData files, and -snapshot-policy=on can theoretically solve the problem of locked files in AppData.

But it doesnt seem to matter for this bug.

If you were to manually invoke Duplicati.GUI.TrayIcon.exe with Run as administrator you’d be “running with privs”. That might allow you to use --snapshot-policy=on and back up some more locked files, but it does not appear to help with this “non-empty blocksets” bug.

Oh and the auto-start with windows starts UN-prived, so be aware of that if you experiment with prived.

ArdieX · February 16, 2019, 6:53am

Hey John,

Sorry for the late reply, I have been busy and haven’t had any time to look into this further yet.

You are correct, I ran my initial backup and received the error. I ran the database repair as suggested in one of the other threads and it indicated no errors found. This was the same job and same backup.

Every subsequent backup simply failed with the mentioned error message, I was only ever able to run that first backup.

If it wasn’t Chrome, I do feel it at least had to do with something else in the \AppData\Local\ folder being backed up. But unfortunately as it seems if I am going to have to take a new full backup I think I will be looking at other products as this one failed out of the gate.

Thanks for your help and I hope any of this information is helpful to someone in the future to get this resolved.