Repair doesnt work


#1

Hi, I got an error which told me I have to repair my database. First I tried the repair but it doesnt solve the problem. After I run delete database and create again and I got this Error:

MainOperation: Repair
RecreateDatabaseResults:
    MainOperation: Repair
    ParsedResult: Success
    EndTime: 01.08.2018 12:42:57 (1533120177)
    BeginTime: 01.08.2018 12:37:11 (1533119831)
    Duration: 00:05:46.3698976
    BackendStatistics:
        RemoteCalls: 227
        BytesUploaded: 0
        BytesDownloaded: 1678274
        FilesUploaded: 0
        FilesDownloaded: 226
        FilesDeleted: 0
        FoldersCreated: 0
        RetryAttempts: 0
        UnknownFileSize: 0
        UnknownFileCount: 0
        KnownFileCount: 0
        KnownFileSize: 0
        LastBackupDate: 01.01.0001 01:00:00 (-62135596800)
        BackupListCount: 0
        TotalQuotaSpace: 0
        FreeQuotaSpace: 0
        AssignedQuotaSpace: 0
        ReportedQuotaError: False
        ReportedQuotaWarning: False
        ParsedResult: Success
ParsedResult: Error
EndTime: 01.08.2018 12:42:57 (1533120177)
BeginTime: 01.08.2018 12:37:11 (1533119831)
Duration: 00:05:46.5271983
Messages: [
    Rebuild database started, downloading 3 filelists,
    Filelists restored, downloading 223 index files,
    Recreate completed, verifying the database consistency,
    Recreate completed, and consistency checks completed, marking database as complete
]
Warnings: []
Errors: [
    Remote file referenced as duplicati-b2a57b1bf0c1d496fb7af7a958508f026.dblock.zip, but not found in list, registering a missing remote file
]

When I now try to run the Backup after I always get the Error:

Failed: Found 1 files that are missing from the remote storage, please run repair
Details: Duplicati.Library.Interface.UserInformationException: Found 1 files that are missing from the remote storage, please run repair
   bei Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)
   bei Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)
   bei Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   bei Duplicati.Library.Main.Controller.<>c__DisplayClass17_0.<Backup>b__0(BackupResults result)
   bei Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

Is there a way to solve the Problem?
I am running on - 2.0.3.3_beta_2018-04-02

Thanks in advance


#2

Did you try running the Repair (not recreate) again?

It’s possible one of the 223 index files mentioned in the messages was referencing the missing dblock file so when the database was recreated (using those index files) it’s assuming the file is there.

(Note that I edited your post by adding “````json” before and “```” after your log and “~~~” before and after your error message to make them easier to read.)


#3

Hi, yes I also runned the repair after again - it doesnt matter. Still the same problem :-/

Fatal error
Duplicati.Library.Interface.UserInformationException: Found 1 files that are missing from the remote storage, please run repair

bei Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)

bei Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)

bei Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)


#4

I suspect setting --no-backend-verification=true will stop the error from happening, but it does this basically by ignoring the problem - so it’s not really a fix.

My GUESS is that deleting all the dlist files from the destination then doing a database Recreate will actually resolve the issue, but that’s a pretty aggressive / hands-on thing to do.

Unfortunately, I don’t know of any other way to fix this issue at the moment. :frowning:


#5

I tried, but when the dlist files are deleted, the database recreation doesnt work because of missing dlist files.
Did I steps in the false order or could you tell me step by step what I shoudt try?


#6

That doesn’t sound right - if dlist files are missing Dulicati should start downloading the larger dblock files to rebuild what would have been in the missing dlist files.

Can you post the actual error message you’re getting?


#7

Are you sure that .DLIST files can be safely deleted from the backend? I’m not sure if Duplicati can distil a file and folder list from raw .DBLOCK data.

Could it be that you meant .DINDEX files instead of .DLIST files? I had a similar problem, deleting all .DINDEX files and recreating the database fixed it for me.


#8

I couldn’t find my original post so I tested the database Repair process again (using 2.0.3.9 canary) on a very small (200M source) backup with the following results:

  • Deleting all 17 .dindex files and running REPAIR resulted in 17 fresh (dated today) .dindex files being created in just over 1 minute
  • Deleting all 18 .dlist files and running REPAIR resulted in 18 fresh (dated today) .dlist files being created in just over 1 minute (though ONE file was flagged with a warning of being uploaded with size X but should be size Y)
  • Deleting all 17 .dindex AND all 18 .dlist files and running REPAIR resulted in 35 fresh (dated today) files being created in 1.5 minutes

So I’d say yes - at least with 2.0.3.9 it is safe to delete all .dindex and/or .dlist files as they can be recreated from the local database ASSUMING the local database is in good condition.

My GUESS is that even if a user has no local database it would still be safe to do the tests I just did, they would simply take longer as all the .dblock files would have to be downloaded and parsed to recreate the missing .dlist and/or .dindex files.

Let me know if you’d like me to test that as well.


#9

I deleted all dlist files and tried to run the repair again. Since more than 24 hours no changes - the progressbar is on 10% and says Starting…
I dont know how to speed up the process - maybe something went wrong and I should cancel and retry :-/