]`
Your run looks like [Using the Command line tools from within the Graphical User Interface](https://docs.duplicati.com/en/latest/03-using-the-graphical-user-interface/#using-the-command-line-tools-from-within-the-graphical-user-interface) where you changed the command from the dropdown. Originally it was probably `backup` so it would use `Commandline arguments` for the source folders you chose. Clear those for `list-broken-files`.
[Recovering by purging files](https://docs.duplicati.com/en/latest/08-disaster-recovery/#recovering-by-purging-files) has an example of what the command (as one line) should look like.
> [@surust](#):
>
> 2nd attempt:
I suppose you did it correctly this time, otherwise it would have failed the same way as first time.
You must have run a Repair at some point, and it gave some warnings or errors. Is that correct?
Maybe not? 2nd attempt might not be `list-broken-files`, as it says “Backup”. Was it backup?
Question still holds though. Was there a repair? What sort of warnings or errors did it show you?
> [@surust](#):
>
> ErrorID: DatabaseRepairInProgress
If it errors out, it says that and certain other things don’t work, but what was it complaining about?
Sometimes `Recreate` can clean things up, but I can’t say much more without knowing messages.
Maybe this is them, although the context of the message wasn’t given, so guessing from the text:
> [@surust](#):
>
> Recreated database has missing blocks and 1 broken filelists. Consider using “list-broken-files” and “purge-broken-files” to purge broken data from the remote store and the database
This is the kind of thing that a silently failed storage of default 50 MB file with `dblock` in the name would cause. The dblock file contains actual source data, meaning some source file backups are damaged. The purge-broken-files removes such broken files, after a list-broken-files shows them.
Problem is that the database is needed to do checks, and you don’t have an intact database now.
The root cause of all of this might be the unreliable Windows WebDAV. What is the server, and is there an unreliable Internet connection in between? Are there any other access ways you can try?
If you’re stuck on WebDAV, then at least you could try telling Duplicati that you’re using WebDAV.
> [@surust](#):
>
> the next time I synchronize, I get various error messages again.
As always, messages are needed, preferably right at the initial failure rather than after other work.
Your [job log](https://docs.duplicati.com/en/latest/03-using-the-graphical-user-interface/#viewing-the-log-files-of-a-backup-job) might have caught some, or maybe About → Show log → Stored and go by its times.
> [@surust](#):
>
> “file://Z:\Backup\Duplicati\Fotos?auth-username=xxx&auth-password=xxxx”
Could be modified to an empty folder, and used with [Duplicati.CommandLine.BackendTester.exe](https://docs.duplicati.com/en/latest/07-other-command-line-utilities/#duplicaticommandlinebackendtesterexe).
If that doesn’t hold up, then you have a problem somewhere outside of Duplicati, on your Z: drive. Changing Duplicati to use WebDAV directly may or may not help things. Depends on failure point.
Using Duplicati with a consistently unreliable destination is likely to get in trouble. Please focus on test and repair, or some alternative storage. Duplicati can be no more reliable than your storage…
---
### Author:  [@surust](https://forum.duplicati.com/u/surust)
#### Post date: [July 20, 2024, 10:59am UTC](https://forum.duplicati.com/t/no-more-nerves-for-duplicati/18681/3 "2024-07-20T10:59:21Z")
First of all, thank you for the detailed answer. I probably don’t understand everything (technically).
The online storage has no other option than WebDav (except via Broswer)
I have a stable Internet connection. Sometimes Windows shuts down when it takes a long time or updates. Or runs out of battery :(. Or I don’t have time to wait any longer and switch off the laptop. But I think the program must be able to correct this itself!
Yes, I run the Graphical User Interface, I don’t know how else. Why are the fields not deleted immediately when a command is selected? I have now deleted the filters and so on, seems to run through. But it doesn’t solve the problem!
Here is everything I have tried:
Repair:
The operation Repair has failed with error: The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes =\> The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
ErrorID: MissingDblockFiles
The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
Return code: 100
Backup:
Error while running Fotos neu
Found 2 files that are missing from the remote storage, please run repair
Purge:
Missing file: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
Missing file: duplicati-ibb3819b50c674446b0ee9c3d6837d459.dindex.zip.aes
Found 2 files that are missing from the remote storage, please run repair
The operation PurgeFiles has failed with error: Found 2 files that are missing from the remote storage, please run repair =\> Found 2 files that are missing from the remote storage, please run repair
ErrorID: MissingRemoteFiles
Found 2 files that are missing from the remote storage, please run repair
Return code: 100
Repair:
The operation Repair has failed with error: The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes =\> The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
ErrorID: MissingDblockFiles
The backup storage destination is missing data files. You can either enable `--rebuild-missing-dblock-files` or run the purge command to remove these files. The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
Return code: 100
I create this 2 Files an run repair again
Repair:
remote file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes is listed as Verified with size 0 but should be 52337501, please verify the sha256 hash “X1x5vbHmsOnV63NWqYf8/UN6FHjwH6y0pg188qgEBrs=”
Downloading file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes (unknown) …
Downloading file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes (unknown) …
Downloading file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes (unknown) …
Downloading file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes (unknown) …
Downloading file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes (unknown) …
Failed to perform verification for file: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes, please run verify; message: Invalid header marker =\> Invalid header marker
Uploading file duplicati-ibb3819b50c674446b0ee9c3d6837d459.dindex.zip.aes (541 bytes) …
Return code: 0
verify:
remote file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes is listed as Verified with size 0 but should be 52337501, please verify the sha256 hash “X1x5vbHmsOnV63NWqYf8/UN6FHjwH6y0pg188qgEBrs=”
Downloading file duplicati-20240712T085556Z.dlist.zip.aes (66,40 KB) …
Downloading file duplicati-ic505f57c04434026b4018656d01dde6d.dindex.zip.aes (33,90 KB) …
Downloading file duplicati-b5135592515914655827a464a2b74a15a.dblock.zip.aes (49,95 MB) …
Examined 3 files and found no errors
Return code: 0
run backup again:
2024-07-15 10:21:49 +02 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes is listed as Verified with size 0 but should be 52337501, please verify the sha256 hash “X1x5vbHmsOnV63NWqYf8/UN6FHjwH6y0pg188qgEBrs=”
2024-07-15 10:22:03 +02 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes is listed as Verified with size 0 but should be 52337501, please verify the sha256 hash “X1x5vbHmsOnV63NWqYf8/UN6FHjwH6y0pg188qgEBrs=”
verify
remote file duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes is listed as Verified with size 0 but should be 52337501, please verify the sha256 hash “X1x5vbHmsOnV63NWqYf8/UN6FHjwH6y0pg188qgEBrs=”
Downloading file duplicati-20240715T075709Z.dlist.zip.aes (1,87 MB) …
Downloading file duplicati-i2b570aefd3e74e389c88b4bdcff78b82.dindex.zip.aes (37,50 KB) …
Downloading file duplicati-b6b61b6eb387d4583b4cbfeef01adc4a8.dblock.zip.aes (49,93 MB) …
Examined 3 files and found no errors
Return code: 0
repair directly in duplicati (without command):
2024-07-15 10:50:56 +02 - [Error-Duplicati.Library.Main.Operation.RepairHandler-RemoteFileVerificationError]: Failed to perform verification for file: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes, please run verify; message: Invalid header marker
CryptographicException: Invalid header marker
This is a never ending story!
In the end, I deleted the backup and started again. After a week, the same error again:
2024-07-20 12:44:51 +02 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-b4d2586fcce284e9fa10e1b504a119bf1.dblock.zip.aes is listed as Uploaded with size 0 but should be 52362253, please verify the sha256 hash “kuYrmTor23d1TqDJrceLby7TVGgG1feRnswrcvxT9uc=”
2024-07-20 12:45:12 +02 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-b4d2586fcce284e9fa10e1b504a119bf1.dblock.zip.aes is listed as Uploaded with size 0 but should be 52362253, please verify the sha256 hash “kuYrmTor23d1TqDJrceLby7TVGgG1feRnswrcvxT9uc=”
So Duplicati is not a program you can trust for your backups!
---
### Author:  [@ts678](https://forum.duplicati.com/u/ts678)
#### Post date: [July 20, 2024, 7:12pm UTC](https://forum.duplicati.com/t/no-more-nerves-for-duplicati/18681/4 "2024-07-20T19:12:06Z")
> [@surust](#):
>
> But I think the program must be able to correct this itself!
It should and usually does. It should clean up the half-done stuff, and continue doing backup.
If you doubt this, you can certainly try it on purpose, but it’s not 100% without possible [issues](https://github.com/duplicati/duplicati/issues).
I’ve killed Duplicati abruptly thousands of times, and some of the findings are in the list above.
I’m not testing with your WebDAV though, and I said some things on it before, and will repeat.
> [@surust](#):
>
> Why are the fields not deleted immediately when a command is selected?
Especially for the options, some are essential or desirable to the command that you are doing.
That argument is weaker for the top boxes such as Target URL and Commandline arguments, however the GUI Commandline experience is similar to what a text-based commandline gives.
[Using Duplicati from the Command Line](https://docs.duplicati.com/en/latest/04-using-duplicati-from-the-command-line/)
and if you’re trying to match a GUI job, you do [Export As Command-line](https://docs.duplicati.com/en/latest/03-using-the-graphical-user-interface/#exporting-a-backup-job-configuration) for the `backup` export, changing the command and what follows it by following the directions for it on the page above.
If you don’t do it that way, there is a good chance you’ll forget something, and have to struggle.
Perhaps GUI support will improve someday, but for now some things need a commandline run.
There are possibly some things that would make life a little easier at smaller development work (because developers are always in short supply), such as giving a link to the user manual page.
> [@surust](#):
>
> The following files are missing: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
This means the local [database](https://docs.duplicati.com/en/latest/03-using-the-graphical-user-interface/#database-management) is not seeing a file it expects. Seeing deeper history of that file is possible in the database, but it would need your posting a link to a [bug report](https://docs.duplicati.com/en/latest/03-using-the-graphical-user-interface/#creating-a-bug-report). Let’s wait on that.
The rebuild suggestion is usually not very effective, so let’s hold off and see what else happens.
> [@surust](#):
>
> Found 2 files that are missing from the remote storage, please run repair
I was going to ask for About → Show log → Live → Warning for names, but they might be below.
> [@surust](#):
>
> Purge:
> Missing file: duplicati-bcdc73bf0f0904a31b78b1674d333c586.dblock.zip.aes
> Missing file: duplicati-ibb3819b50c674446b0ee9c3d6837d459.dindex.zip.aes
Was that actually [The PURGE command](https://docs.duplicati.com/en/latest/04-using-duplicati-from-the-command-line/#the-purge-command)? Or was it [The PURGE-BROKEN-FILES command](https://docs.duplicati.com/en/latest/04-using-duplicati-from-the-command-line/#the-purge-broken-files-command)?
I think the message intended the latter (probably a bug). Anyway, those might be the two files.
Maybe a developer will stop by to confirm the typo and fix it, and advise on recovery methods.
[Recovering by purging files](https://docs.duplicati.com/en/latest/08-disaster-recovery/#recovering-by-purging-files) is optional `list-broken-files` and then a `purge-broken-files` which deletes the backups of any source file versions that have data in the missing dblock file.
How it went missing is a different question. Duplicati does rely on a destination not losing files. Offhand, I know of one Duplicati bug where an interruption at the wrong time can make it think there’s a missing dindex file (false alarm), but I don’t think I know of a false positive on dblocks.
Especially after your second failure after another week, some upload issue might actually exist.
> [@surust](#):
>
> Repairing or deleting/restoring the local database also takes a very, very long time.
Do you have the job log from the last Recreate? If so, the Complete log might hold some clues. Perhaps you remember something also? Any messages? Were you watching its progress bar?
The last 10% can become slow if the backup destination is missing blocks and needs a search.
What’s supposed to happen is the dindex files locate everything, but you’re missing one dindex.
To tell more about the history and importance of the missing files would need the DB bug report.
> [@surust](#):
>
> I create this 2 Files
You can’t just toss files into the destination and think that Duplicati will think they are proper files.
Duplicati checks the backup, and is complaining. This is expected when the destination is wrong.
> [@surust](#):
>
> After a week, the same error again:
Well, no, or at least not that you posted anywhere here. It looks like you got a different error, right?
This one looks like a recently uploaded default 50 MB file is now size 0. Can you confirm the size?
I still don’t like your using a Windows drive letter, as mentioned above. Are you still doing that? It’s another place to lose data, and it’s harder for Duplicati to notice if Windows WebDAV is losing files.
[WebDAV](https://docs.duplicati.com/en/latest/05-storage-providers/#webdav) is probably the safer choice for using WebDAV on Duplicati, to avoid Windows problems.
Do you have any idea now (maybe you’re paying attention?) to whether there was any interruption of the previous backup? Knowing that would help to figure out if interruptions are causing troubles.
You can check your job logs and also the server log at About → Show log → Stored for their clues.
Having a [log-file](https://docs.duplicati.com/en/latest/06-advanced-options/#log-file)=\ at [log-file-log-level](https://docs.duplicati.com/en/latest/06-advanced-options/#log-file-log-level)=information would have gotten information on second failure. For example, we could easily see interrupted backups, and also the filenames that got lost.
Currently, that chance is lost, but you can still do About → Show log → Live → Warning and push `Verify files` button to get missing file names (might need click on line). If one dblock, it’s easier compared to last time, and you can probably do `list-broken-files` and `purge-broken-files`.
Files should not be breaking, and I still suspect that your Z: drive plan is adding extra areas to fail.
> [@ts678](#):
>
> > [@surust](#):
> >
> > “file://Z:\Backup\Duplicati\Fotos?auth-username=xxx&auth-password=xxxx”
>
> Could be modified to an empty folder, and used with [Duplicati.CommandLine.BackendTester.exe](https://docs.duplicati.com/en/latest/07-other-command-line-utilities/#duplicaticommandlinebackendtesterexe).
> If that doesn’t hold up, then you have a problem somewhere outside of Duplicati, on your Z: drive. Changing Duplicati to use WebDAV directly may or may not help things. Depends on failure point.
---
### Author:  [@ts678](https://forum.duplicati.com/u/ts678)
#### Post date: [July 20, 2024, 9:12pm UTC](https://forum.duplicati.com/t/no-more-nerves-for-duplicati/18681/5 "2024-07-20T21:12:28Z")
By the way, Microsoft also advises against using their WebDAV client. It’s already deprecated
[Deprecated features for Windows client](https://learn.microsoft.com/en-us/windows/whats-new/deprecated-features)
[Inquiry about the Deprecation of WebClient/WebDAV Services, Overall Impact, and Replacement Options](https://learn.microsoft.com/en-us/answers/questions/1609470/inquiry-about-the-deprecation-of-webclient-webdav) are people guessing why that is.
On the WebDAV server, are there any logs that would have information on the lost/empty files?
---
### Author:  [@ts678](https://forum.duplicati.com/u/ts678)
#### Post date: [July 21, 2024, 12:28am UTC](https://forum.duplicati.com/t/no-more-nerves-for-duplicati/18681/6 "2024-07-21T00:28:56Z")
[Using the WebDAV Redirector](https://learn.microsoft.com/en-us/iis/publish/using-webdav/using-the-webdav-redirector) shows a 50,000,000 decimal `FileSizeLimitInBytes`.
You can see if you can see a larger one, but one that failed to upload was 52,337,501.
I don’t know what size your normal backups are, but you can figure it out by file dates.
Even if your backup updates are small, I’d have thought initial backup would be large.
[Zero byte files](https://sabre.io/dav/0bytes/) possibly explains those, where that’s what some clients do. Windows?
[WebDAV API Functions](https://learn.microsoft.com/en-us/windows/win32/webdav/webdav-api-functions) is maybe similar to what the drive letter you use has available.
You can see the talk of having to flush local files to the server. What if it’s interrupted?
[Webdav upload fast but hangs at 99% until it eventually finishes](https://help.nextcloud.com/t/webdav-upload-fast-but-hangs-at-99-until-it-eventually-finishes/102280/12) said Windows caches.
Personally, I’m not positive, but if you’re not using Duplicati WebDAV, I’d prefer you did.