I am attempting to backup data to Mailfence who use Webdav.
I entered the details and the test function says working.
On a run it looks to be uploading but then ends with:
One or more errors occurred. (Unable to read data from the transport connection: The socket has been shut down
What can I troubleshoot?
Thanks!
Working after repair!
Do you mean Duplicati Repair button (which would be surprising), or something else? What?
The last time somebody hit this, I think they had Options Remote volume size set too large.
Default is 50 MB. Make it too big, and the other end gets unhappy. Other bad things happen:
Yes, I even tried 20 MB and it made no difference.
Every new backup I make seems to need a Duplicati repair job (database) done before it will sync!
This should not happen unless the prior backup fails, and even then it might not need that.
Please provide any messages you get, including job log and About → Show log → Stored.
If needed, additional logging can be started to try to improve view of what’s going on there.
Why was this closed as “Working after repair!”?
User-level troubleshooting tools would include:
Duplicati.CommandLine.BackendTool.exe
Duplicati.CommandLine.BackendTester.exe
You can start with a URL from Export As Command-line, probably edited to use a test folder.
Hmm. on a second run I now get:
2024-10-13 13:20:01 +00 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Repair has failed with error: No files were found at the remote location, perhaps the target url is incorrect?
UserInformationException: No files were found at the remote location, perhaps the target url is incorrect?
In bout → Show log → Stored
Duplicati.Library.Interface.UserInformationException: No files were found at the remote location, perhaps the target url is incorrect?
at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun (Duplicati.Library.Main.Database.LocalDatabase dbparent, System.Boolean updating, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00132] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run (System.String path, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00037] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal (Duplicati.Library.Utility.IFilter filter) [0x000ba] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Operation.RepairHandler.Run (Duplicati.Library.Utility.IFilter filter) [0x00012] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Controller+<>c__DisplayClass18_0.<Repair>b__0 (Duplicati.Library.Main.RepairResults result) [0x0001c] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0036c] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Library.Main.Controller.Repair (Duplicati.Library.Utility.IFilter filter) [0x0001a] in <9b99db820df3422db0aa96d4648dc7b1>:0
at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x003ad] in <17916a3bd21d425da6aeece2c5428e16>:0
Again on another run of another backup
Found 20 files that are missing from the remote storage, please run repair
This indicates that the file listing you get is not correct.
It could be some limitation that prevents Duplicati from listing all files. Can you see how many files Duplicati expects and how many are present?
How do I find that info?
Complete logs
{
"MainOperation": "Test",
"VerificationsActualLength": 0,
"Verifications": [],
"ParsedResult": "Fatal",
"Interrupted": false,
"Version": "2.0.8.1 (2.0.8.1_beta_2024-05-07)",
"EndTime": "2024-10-13T13:38:06.655259Z",
"BeginTime": "2024-10-13T13:38:05.600549Z",
"Duration": "00:00:01.0547100",
"MessagesActualLength": 3,
"WarningsActualLength": 20,
"ErrorsActualLength": 2,
"Messages": [
"2024-10-13 13:38:05 +00 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Test has started",
"2024-10-13 13:38:06 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2024-10-13 13:38:06 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: ()"
],
"Warnings": [
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-ie634a66f199c4531a60e4109d09cefdb.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-20241013T102159Z.dlist.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b450f5b96e4594849b60bd9479040b08f.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b247e4bfebb59415a8f3af2d62768ce96.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i2759c456113e43189e38fc7818884f9b.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-20241013T110515Z.dlist.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-bb7b538b7438a4b0cb97ea479580c1757.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b415502dac6a5466d83aa1b36e457d93b.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i4f43039c70f84af498a5d850da38c19d.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b92cab73ccd9e452392e85bbd49fdd400.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i261f4fce0b834b91b58a9e6c5aa601f2.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-be0f0fd10030e4ad6bd9245e72e0b7783.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i44b338e470aa4430ad8f2bafeebda927.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b5dfec33eb7a543fdb3d3b44a71fa54dd.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i8122eb5610b64afbb92c60de43012cd6.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-b719023135e3444e791b1114d5a1cd2d6.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-id84417cc44a745deb21959d2be84b7ad.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-bacfba65072ac4e64b814037d2068ae8f.dblock.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-ia4b0986939f141fa86c7aee217432887.dindex.zip.aes",
"2024-10-13 13:38:06 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingFile]: Missing file: duplicati-i8b7de3e18eca41a68b0b696eeddd651f.dindex.zip.aes"
],
"Errors": [
"2024-10-13 13:38:06 +00 - [Error-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteFiles]: Found 20 files that are missing from the remote storage, please run repair",
"2024-10-13 13:38:06 +00 - [Error-Duplicati.Library.Main.Controller-FailedOperation]: The operation Test has failed with error: Found 20 files that are missing from the remote storage, please run repair\nRemoteListVerificationException: Found 20 files that are missing from the remote storage, please run repair"
],
"BackendStatistics": {
"RemoteCalls": 1,
"BytesUploaded": 0,
"BytesDownloaded": 0,
"FilesUploaded": 0,
"FilesDownloaded": 0,
"FilesDeleted": 0,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 0,
"KnownFileSize": 0,
"LastBackupDate": "0001-01-01T00:00:00",
"BackupListCount": 2,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Test",
"ParsedResult": "Success",
"Interrupted": false,
"Version": "2.0.8.1 (2.0.8.1_beta_2024-05-07)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-10-13T13:38:05.600554Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
}
would be (maybe) a list problem. In addition to previously named tools, WebDAV offers:
--debug-propfind-fileTo aid in debugging issues, it is possible to set a path to a file that will be overwritten with the PROPFIND response
Of course, user actions can cause an empty destination, but I assume you didn’t do such.
is kind of odd though. Anyway, you have someone else in on this (thanks!) so I may back out.
but to clarify, somehow Duplicati expects no files. Can you actually see what’s on destination?
Duplicati job → Show log → Remote will often end with a list. Click that to see what it saw.
Please also consider testing the destination as suggested.
I can see in the log that it lists the filenames that it expects. Can you manually inspect the destination and see if the files are there?
Nicely spotted @ts678 !
I think this indicates that Duplicati does not see any files for some reason. Can you try the propfind debug step mentioned above?
The files were there at Mailfence - but checking just now they are ALL gone!! 
If Mailfence did it, there’s probably no record. If Duplicati did it (though I don’t know why), there would be a record in job → Show log → Remote and flip until you see some delete happening.
Alternatively (avoids flip but is more technically challenging for some), you can look right in the database with DB Browser for SQLite or similar (assuming it hasn’t been recreated since then).
EDIT 1:
Example:
however I’m not sure why Duplicati would be doing delete. You can look if you suspect though. Delete operations above were probably part of retention policy version deletes and are normal.
EDIT 2:
Example of GUI-provided record which you possibly will have to flip back through to right place:
Above delete are probably a result of Compacting files at the backend, and so are also normal.
1 Like
Ok, thanks.
I have just emailed Mailfence.
It would seem very strange for Duplicati to delete the files.
This is a new backup location.
Duplicati never had a problem backing up to my homeserver - apart from the fact it is currently offline. I need to look into that!
It would leave records, as I’ve been adding to my above post through a series of edits.
Completely removing everything would need a very specific set of odd circumstances.
Testing with the suggested tools is still suggested.
Ok, I am going to have to leave it for now. It is 1.30 AM.
I need to sleep! I cannot focus.
Thanks for the help!
I will go over your writings tomorrow.
Cheers!
The only operations that would delete files are compact and repair.
Compact will not remove the last backup, and repair will not sync an empty local backup.
Hope there is an explanation from the storage provider.
Hi ts678, for your information, the --debug-propfind-file for webdav has been deprecated. It was originally there for debugging purposes during development, would having it back be important on future versions?