Good evening, this weekend during the duplicati backup process it was necessary to stop the process for the execution of a task and the duplicati ended up breaking the backup in the destination, giving the error:
Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
I tried deleting the local database and rebuild it after trying to run it multiple times and to no avail, now the error it is giving is:
Unexpected number of remote volumes marked as deleted. Found 0 filesets, but 11 volumes
Does anyone know how to resolve so I do not have to do another full backup and lose all versioning so far?
PS: I’m using OneDrive as the destination for this backup.
When the problem occurred I was using version 2.0.3.2, however I took advantage to upgrade to Canary 2.0.3.5-1 and when I ran the gui Recreate (delete and repair) web option the problem of not being able to run the backup occurred
Hmm, the error suggests that the filelists are missing from the backend. Does there appear to be any .dlist files when you look at the remote store manually?
There are several .dlist.zip.aes files and I am currently running version 2.0.3.5_canary_2018-04-13, updated after the problem occurred (as I thought that updating could resolve the error).
Running 2.0.3.5_canary_2018-04-13 with Minio and B2 as backend.
1st, I had:
Errors: [
2018-04-29 20:01:21 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-20180430T000000Z.dlist.zip.aes,
2018-04-29 20:02:12 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-ib24ac61eed1b4353ab726737c5997496.dindex.zip.aes,
2018-04-29 20:03:03 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-ba1eadafe92b04631ad00657efd2c75c1.dblock.zip.aes
]
TestResults:
MainOperation: Test
Verifications: [
Key: duplicati-20180430T000000Z.dlist.zip.aes
Value: [
Key: Error
Value: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
],
Key: duplicati-ib24ac61eed1b4353ab726737c5997496.dindex.zip.aes
Value: [
Key: Error
Value: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
],
Key: duplicati-ba1eadafe92b04631ad00657efd2c75c1.dblock.zip.aes
Value: [
Key: Error
Value: Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.
]
]
I tried to repair DB, with the error still happen. But at least the backup was still working.
Then I tried to rebuild the DB, but now the backup failed each time with:
Unexpected number of remote volumes marked as deleted. Found 2 filesets, but 20 volumes
Did this start for you after cancelling a running backup or did the error messages just start up (in which case I may move your stuff to their own post)?
Did you try a repair (NOT another rebuild) of your DB after the rebuild? It seems that there are instances where a rebuilt fixes some stuff, but then a repair is needed to resolve other issues for some reason.
This happens during the destination cleanup process, likely called by a retention rule.
More specifically, records that are no longer needed are deleted from one table then those deleted records are used to update another table to flag specific remote files for deletion (which happens in a later step).
The problem being reported is that the number coming out of the delete doesn’t match the number coming out of the update. In your case 5 records were deleted but 7 files were flagged for deletion.
My guess is something else happened to flag additional files for deletion but they’re being caught up in this cleanup step causing the mismatch.
Unfortunately, I’m not sure what the “correct” step is to recover from this - I could suggest another repair, but I have no idea if it would actually help or not. Perhaps @Pectojin might have a suggestion (and if applicable, maybe we could add it to the error message for future use).
Hmm, I’m not very familiar with the delete process. The issue seems to be tracked a couple of places, but I can’t find any solutions for it.
Maybe --auto-cleanup can help here?
If a backup is interrupted there will likely be partial files present on the backend. Using this flag, Duplicati will automatically remove such files when encountered.
