Missing data file on my backup storage destination

Support
1

Hi guys,
I need your help again…
My working backup routine stopped working. :frowning:

Error: The backup storage destination is missing data files. You can either enable --rebuild-missing- ***d'block*** -files or run the purge command to remove these files. The following files are missing… It’s only one file…

What I’ve tried so far:

  • Repair Database: Error message that translates to “Network path not found”
  • Command line " list-broken-files": After several minutes of “Listing remote folder …” I get an error message that translates to “Network path not found”. The path is correct and can be accessed by duplicati. How else could it find one broken file?
  • Command line “purge-broken-files”: same as with the other tries “Network path not found”

What you should know:

  • I’m accessing a remote storage via WebDAV
  • The database has about 900 GB data
  • Listing the folders and files on windows explorer is really slow
  • I’ve made some changes to the registry to raise the timeout limits
  • The backup worked some time ago with nothing changed

Do you have some advice on what to do next?

Thanks in advance
Seb

2

I would just review the logs what goes exactly wrong with the WebDAV. Sounds like a typical path / authentication error. Based on your post can’t come up with any better solutions or something to try. I assume you use separate authentication / login for Duplicati & your normal WebDAV browsing, so it would be typical to have configuration mismatch in that case.

You mean the backup storage size is 900 GB? Or the Duplicati database is 900 GB (which hopefully isn’t true). Technically the Duplicati block storage is just “flat files” and not a database at all.

If you can isolate the exact error from debug log, and can’t solve situation based on it yourself, then provide the error message for further analysis.

3

Thanks for your reply!
The whole backup has 900 GB, not the database - sorry :slight_smile:

I’ve tried to repair the database and this is the error out of the debug log:

Sep 14, 2019 11:57 AM: The operation Repair has failed with error: The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-b3722facd53ae42cb98f438d137af4df1.dblock.zip.aes
{“ClassName”:“Duplicati.Library.Interface.UserInformationException”,“Message”:“The backup storage destination is missing data files. You can either enable --rebuild-missing-dblock-files or run the purge command to remove these files. The following files are missing: duplicati-b3722facd53ae42cb98f438d137af4df1.dblock.zip.aes”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" bei Duplicati.Library.Main.Operation.RepairHandler.RunRepairRemote()\r\n bei Duplicati.Library.Main.Operation.RepairHandler.Run(IFilter filter)\r\n bei Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)\r\n bei Duplicati.Library.Main.Controller.Repair(IFilter filter)\r\n bei Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nRunRepairRemote\nDuplicati.Library.Main, Version=2.0.4.23, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.Operation.RepairHandler\nVoid RunRepairRemote()”,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}

Now I’ll try to purge this file…

Finished!

Listing remote folder …
Uploading file (18,80 MB) …
Deleting file duplicati-20190609T143937Z.dlist.zip.aes …
Return code: 0

Okay, why did it work today… Now I’ll try to do a full backup again.

Edit2:
Oh no… Now the network path cannot be found:

Sep 14, 2019 12:45 PM: Failed while executing “Backup” with id: 2
System.IO.IOException: Der Netzwerkpfad wurde nicht gefunden.

bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)

bei System.IO.DriveInfo.get_TotalSize()

bei Duplicati.Library.Backend.File.get_Quota()

bei Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)

bei Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)

bei Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)

bei Duplicati.Library.Main.Operation.BackupHandler.d__19.MoveNext()

— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —

bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

bei CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

bei Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.b__0(BackupResults result)

bei Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)

bei Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)

bei Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

What now?

Edit3
Now it’s working… But I forgot to exclude one folder.
So I stopped the backup, added the exclusion.
But now I’ve got an even bigger problem when I’m trying to start the backup again:

Sep 14, 2019 2:31 PM: The operation Backup has failed with error: Detected non-empty blocksets with no associated blocks!
{“ClassName”:“System.Exception”,“Message”:“Detected non-empty blocksets with no associated blocks!”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" bei Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency(Int64 blocksize, Int64 hashsize, Boolean verifyfilelists, IDbTransaction transaction)\r\n bei Duplicati.Library.Main.Operation.Backup.BackupDatabase.<>c__DisplayClass32_0.b__0()\r\n bei Duplicati.Library.Main.Operation.Common.SingleRunner.<>c__DisplayClass3_0.b__0()\r\n bei Duplicati.Library.Main.Operation.Common.SingleRunner.d__21.MoveNext()\r\n--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()\r\n--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n bei Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)\r\n bei Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)\r\n bei Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)\r\n bei Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nVerifyConsistency\nDuplicati.Library.Main, Version=2.0.4.23, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.Database.LocalDatabase\nVoid VerifyConsistency(Int64, Int64, Boolean, System.Data.IDbTransaction)”,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}

Repair does not help either… HELP PLEASE!

4

I can feel your pain mate. So sorry to hear about your problems. I think I’ve seen everything except the network path not found error several times. And I’ve whined about the similar unfortunate problems as you’re experiencing.

There are several threads how to fix the non-empty blocksets problem. Yet I think personally I never did that, because there wasn’t any easy and simple way to get it done. In my case, luckily the backup sets were so small that it was trivial for me to just store the old backup for a few months and start the primary backup from fresh. But for some of my backups in terabyte range, I really were hoping to be able to get the state restored without full re-upload.

Also if you’re not using latest version of Duplicati, do update. Because these problems used to occur lot more often previously than with newer versions.

I hope you’ll get that fixed, but if you’re in hurry, just stash the old backups for a while and restart fresh, much faster and lot less pain to go through.

5

Thanks for your advice.
A rebuild of the local database seems to solve my last problem.
Was quite fast though…
A complete fresh backup is not an option as I have to send 900GB over my internet connection which would take weeks to complete.
I’m on the latest version of duplicati. Hopefully the backup job runs successfull now…

6

If you’re using WebDAV via Windows giving you a virtual file system, this would seem to be a Windows issue. Google search for "webdav" "network path * could not be found" is finding a few cases. Although it makes me nervous to invite similar issues in Duplicati, have you tried WebDAV by Duplicati?

v2.0.4.28-2.0.4.28_canary_2019-09-05 has a fix for a case of Detected non-empty blocksets with no associated blocks! caused by a file being written while being backed up. I don’t know if that’s your case. You could certainly test canary if you like. They’re sometimes risky, but this one is behaving itself (so far). Changing your Settings back to Beta channel might be a good idea if you decide to hand-pick this canary.

Fixing the problem once it’s happened is left to old methods, including Database Recreate which also got fixed in v2.0.4.18-2.0.4.18_canary_2019-05-12 for a severe performance bug. Again, fix is not yet in Beta, however if you want this fix and not the other one, v2.0.4.21-2.0.4.21_experimental_2019-06-28 has it and you don’t have to touch the bleeding-edge Canary channel. Experimental at the moment is like a pre-Beta.

1 Like
7

Thanks for your advice, too.
After 9,5 hours my backup job completed without any (new) problems :slight_smile:

I’m on beta-channel right now. If the issues return, I’ll try to go for the canary releases!