Backup repair fails

Support
1

I could use a little help with a backup I’ve been running on some hyper-v vms. The backup destination is a windows network share. I’ve tried running a repair, and deleting and recreating the database, but nothing I’ve tried has resolved the problem.
The errors in the log include Failed while executing “Repair” with id: 2

System.IO.InvalidDataException: Found inconsistency in the following files while validating database: 
C:\Users\Public\Documents\Hyper-V\Virtual Hard Disks\vm1.vhdx, actual size 20640169984, dbsize 20316160000, blocksetid: 38
. Run repair to fix it.
   at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(IDbTransaction transaction, Int64 blocksize, Int64 hashsize, Boolean verifyfilelists)
   at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run(String path, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   at Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal(IFilter filter)
   at Duplicati.Library.Main.Operation.RepairHandler.Run(IFilter filter)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Repair(IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

Failed while executing “Backup with id” 2

Duplicati.Library.Interface.UserInformationException: Found 371 remote files that are not recorded in local storage, please run repair
   at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

Do you have any advice on how to fix this, or do I need to delete the remote files and start over?

2

I believe this error is saying that the size of vm1.vhdx as recorded in your local database doesn’t match the size as recorded in your destination. This puts your local database into suspect and instructs you to Run repair to fix it which should basically compare the local database against the remote index files and make sure everything matches.

Did you try using the “Repair” button in the “Database…” menu?

This is likely a side effect of the first error in that once Duplicati thinks the local databse might be suspect it stops using it at which point all files on the destination are considered “extra”. It should go away once your database is repaired.

Let us know if that doesn’t work for you. If you haven’t read through them yet, there are also a few other posts with the same error message.

3

Yes, as I mentioned I’ve tried both repairing and deleting and recreating the database. I get the same errors with the new database.

4

Oops, sorry I missed that. @kenkendk is the expert on database issues so hopefully he’ll get a chance to check this post out soon.

6

I’m getting this or a very similar problem, every time I use Duplicati. If I can’t rely on it to keep my data, why would I use it?

7

I agree completely - but there are many users who don’t have any problems at all, so the issue is most likely something related to what you’re doing with Duplicati, how you’re doing it, or what it’s being done on. And we’d love to figure out which of those are the issue(s) so we can try to make sure it works better for you and doesn’t happen to others in the future.

8

I’m on Windows 10, and I’m not doing anything funky with the data itself. It’s “large” - say like 30GB-200GB range - with no junctions or anything weird like that. It’s likely that some of the paths are beyond the 260 character limit. Lastly, I’m storing the data on a VeraCrypt encrypted external hard drive. The drives seem to be fully reliable, they’re not being smashed around or anything, and I’ve never had problems writing to them via. the Windows Explorer, Command Prompt, PowerShell, etc.; I believe it uses a service or driver or something to ensure that all file reads and writes are done with encryption. Every time Duplicati breaks like this on me, I have my drive “unencrypted” for reads and writes with VeraCrypt.

Hopefully that will help you as you try to test/diagnose issues. Sorry, but I’m just going to have to use a different backup system, at least for a long while.

9

Thanks for the additional info - I use VeraCrypt myself and haven’t had any problems with Duplicati, but who knows - maybe there’s a particular setting it doesn’t like.

I hope you find a backup solution that works well for you - we’d rather have you backing up with anything than not backing up at all. If you really like what you find, feel free to let us know what you chose - perhaps we can add it to our Comparison posts. :slight_smile:

10

As @JonMikelV mentions this is caused by an error in the database, and I think the other error is related to this.

I know I fixed an issue in Duplicati at some point that had this error if the file was changed during the backup. What version of Duplicati are you using?

11

That sounds like a problem with “file is being modified while being backed up”.
What version of Duplicati is working like this?

12

I have Duplicati - 2.0.2.1_beta_2017-08-01 installed on the server. The backup has snapshot-policy set to required, so it’s strange that the file would have changed during backup if vss is being used. If I get some free time I might try to get remote debugging set up on the server where it’s running to see if I can track down what is causing the error.