Backup failed due to error: Could not find file "/opt/Duplicati/tmp/dup-

Peter_Strasser · September 15, 2019, 8:35am

Hi guys,
I need your help as my working backup routine stopped working.

I’m using Duplicati - 2.0.4.28_canary_2019-09-05 on a QNAP TS-431P (QTS 4.3.60993).

I have 4 different backup jobs. And since some days with all of them the backup stops after a while with the following error message:

One or more errors occurred. (Could not find file “/opt/Duplicati/tmp/dup-b9cf657d-c3ec-432e-955a-62471afddfc4” (Could not find file “/opt/Duplicati/tmp/dup-b9cf657d-c3ec-432e-955a-62471afddfc4”) (One or more errors occurred. (Connection reset by peer)))

It seems that the tmp files are getting deleted before the upload is finished. How can this happen? I can see that during the backup some tmp files are generated and they can also be seen in the tmp folder. And there should be enough space on the drive.

Do you have some advice on what to do next?

Thanks in advance
Peter

ts678 · September 16, 2019, 1:18pm

Welcome to the forum @Peter_Strasser

Does “should” mean known? If you haven’t measured, you could try df /opt/Duplicati/tmp
Keeping an eye on /tmp might also be good. I found reports about very small ones, e.g. 64 MB.
–tempdir talks about some of the challenges of keeping things out of system default temp area.

Are you using default remote volume size (–dblock-size) of 50 MB? What is at the destination?
“Connection reset by peer” suggests that whatever it is, it closed your TCP connection on you.

Also note that “after a while” the backup moves to Verifying backend files which also uses tmp.

Probably next thing to do is to look at a log for a clearer view of what it’s doing at the error time.

Watching a live log at About → Show log → Live → Retry might be the easiest way to do this.

Adding a –log-file at –log-file-log-level=retry is more setup, but a slightly nicer unattended option.

Peter_Strasser · September 16, 2019, 5:46pm

Hi ts678,

ok, I checked `df /opt/Duplicati/tmp --> [/opt] # df /opt/Duplicati/tmp
Filesystem Size Used Available Use% Mounted on /dev/md0 2.7T 1.1T 1.6T 41% /share/MD0_DATA
So this shouldn’t be the problem.

And yes, I’m using the default remote volume size of 50MB.

As I haven’t changed anything on my configuration I also suspect the backend. I use a WebDav Server from a friend. But he also claims nothing has changed.

I checked the live log with level “retry”. Unfortunately I still have no clue. Maybe you can see something in the log:

16. Sept. 2019 03:05: UsageReporter failed
{“ClassName”:“System.Net.WebException”,“Message”:“Error: SecureChannelFailure (Authentication failed because the remote party has closed the transport stream.)”,“Data”:null,“InnerException”:{“ClassName”:“System.IO.IOException”,“Message”:“Authentication failed because the remote party has closed the transport stream.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x0008e] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)\n at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x000ff] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x0008b] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00252] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x00126] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001ba] in <906205e228b04a3e9ab3c0712759f5c5>:0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146232800,“Source”:“mscorlib”},“HelpURL”:null,“StackTraceString”:” at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x0021a] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x00141] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.WebOperation.Run () [0x0009a] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.WebCompletionSource1[T].WaitForCompletion () [0x00094] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <8b9719de1a0e4ca7936db1e2d5cba406>:0 \n at Duplicati.Library.Utility.AsyncHttpRequest.GetRequestStream (System.Int64 contentlength) [0x0007f] in <8b9719de1a0e4ca7936db1e2d5cba406>:0 \n at Duplicati.Library.UsageReporter.ReportSetUploader+<>c.<Run>b__3_0 (CoCoL.IReadChannel1[T] chan) [0x000c4] in <14c5332eaef745e4ba4402e70732813b>:0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“Duplicati.Library.Utility”}

16. Sept. 2019 03:04: Die Operation Backup ist mit folgenden Fehler fehlgeschlagen: One or more errors occurred. (Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3” (Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3”) (One or more errors occurred. (Connection reset by peer)))
{“ClassName”:“System.AggregateException”,“Message”:“One or more errors occurred.”,“Data”:null,“InnerException”:{“ClassName”:“System.AggregateException”,“Message”:“Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3"”,“Data”:null,“InnerException”:{“ClassName”:“System.IO.FileNotFoundException”,“Message”:“Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3"”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in :0 \n at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in :0 \n at (wrapper remoting-invoke-with-check) System.IO.FileStream…ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)\n at Duplicati.Library.Main.Volumes.VolumeReaderBase.LoadCompressor (System.String compressor, System.String file, Duplicati.Library.Main.Options options, System.IO.Stream& stream) [0x00000] in :0 \n at Duplicati.Library.Main.Volumes.VolumeReaderBase…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00007] in :0 \n at Duplicati.Library.Main.Volumes.BlockVolumeReader…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00010] in :0 \n at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess+<>c__DisplayClass0_0.b__0 (<>f__AnonymousType92[<Input>j__TPar,<Output>j__TPar] self) [0x0023c] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (System.Collections.Generic.IEnumerable1[T] sources, Duplicati.Library.Snapshots.ISnapshotService snapshot, Duplicati.Library.Snapshots.UsnJournalService journalService, Duplicati.Library.Main.Operation.Backup.BackupDatabase database, Duplicati.Library.Main.Operation.Backup.BackupStatsCollector stats, Duplicati.Library.Main.Options options, Duplicati.Library.Utility.IFilter sourcefilter, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.BackupResults result, Duplicati.Library.Main.Operation.Common.ITaskReader taskreader, System.Int64 lastfilesetid) [0x00349] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00a06] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147024894,"Source":"mscorlib","FileNotFound_FileName":"/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3","FileNotFound_FusionLog":null},"HelpURL":null,"StackTraceString":" at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00ffe] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"Duplicati.Library.Main","InnerExceptions":[{"ClassName":"System.IO.FileNotFoundException","Message":"Could not find file \"/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3\"","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in <b5ecc4525d914885a148d4a08ffbf3f3>:0 \n at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in <b5ecc4525d914885a148d4a08ffbf3f3>:0 \n at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)\n at Duplicati.Library.Main.Volumes.VolumeReaderBase.LoadCompressor (System.String compressor, System.String file, Duplicati.Library.Main.Options options, System.IO.Stream& stream) [0x00000] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Volumes.VolumeReaderBase..ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00007] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Volumes.BlockVolumeReader..ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00010] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess+<>c__DisplayClass0_0.<Run>b__0 (<>f__AnonymousType92[j__TPar,j__TPar] self) [0x0023c] in :0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in <ea1db083b4fe4d4a85e24c85e3bce198>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (System.Collections.Generic.IEnumerable1[T] sources, Duplicati.Library.Snapshots.ISnapshotService snapshot, Duplicati.Library.Snapshots.UsnJournalService journalService, Duplicati.Library.Main.Operation.Backup.BackupDatabase database, Duplicati.Library.Main.Operation.Backup.BackupStatsCollector stats, Duplicati.Library.Main.Options options, Duplicati.Library.Utility.IFilter sourcefilter, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.BackupResults result, Duplicati.Library.Main.Operation.Common.ITaskReader taskreader, System.Int64 lastfilesetid) [0x00349] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00a06] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147024894,“Source”:“mscorlib”,“FileNotFound_FileName”:”/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3”,“FileNotFound_FusionLog”:null},{“ClassName”:“System.AggregateException”,“Message”:“One or more errors occurred.”,“Data”:null,“InnerException”:{“NativeErrorCode”:10054,“ClassName”:“System.Net.Sockets.SocketException”,“Message”:“Connection reset by peer”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at System.Net.Sockets.Socket.EndSend (System.IAsyncResult asyncResult) [0x00012] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x00057] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n— End of stack trace from previous location where exception was thrown —\n\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.b__13_0 (<>f__AnonymousType111[<Input>j__TPar] self) [0x00673] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType111[j__TPar] self) [0x0089e] in :0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in <ea1db083b4fe4d4a85e24c85e3bce198>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467259,"Source":"mscorlib"},"HelpURL":null,"StackTraceString":null,"RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":null,"InnerExceptions":[{"NativeErrorCode":10054,"ClassName":"System.Net.Sockets.SocketException","Message":"Connection reset by peer","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.Net.Sockets.Socket.EndSend (System.IAsyncResult asyncResult) [0x00012] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x00057] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType111[j__TPar] self) [0x00673] in :0 \n at Duplicati.Library.Main.Operation.Backup.BackendUploader.b__13_0 (<>f__AnonymousType111[<Input>j__TPar] self) [0x0089e] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147467259,“Source”:“mscorlib”}]}]},“HelpURL”:null,“StackTraceString”:” at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x0005d] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00008] in :0 \n at Duplicati.Library.Main.Controller+<>c__DisplayClass13_0.b__0 (Duplicati.Library.Main.BackupResults result) [0x00035] in :0 \n at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String& paths, Duplicati.Library.Utility.IFilter& filter, System.Action1[T] method) [0x0026f] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Controller.Backup (System.String[] inputsources, Duplicati.Library.Utility.IFilter filter) [0x00074] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00353] in <ee79d5f802c146908ba5accd2b4f4f2e>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233088,"Source":"CoCoL","InnerExceptions":[{"ClassName":"System.AggregateException","Message":"Could not find file \"/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3\"","Data":null,"InnerException":{"ClassName":"System.IO.FileNotFoundException","Message":"Could not find file \"/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3\"","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in <b5ecc4525d914885a148d4a08ffbf3f3>:0 \n at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in <b5ecc4525d914885a148d4a08ffbf3f3>:0 \n at (wrapper remoting-invoke-with-check) System.IO.FileStream..ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)\n at Duplicati.Library.Main.Volumes.VolumeReaderBase.LoadCompressor (System.String compressor, System.String file, Duplicati.Library.Main.Options options, System.IO.Stream& stream) [0x00000] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Volumes.VolumeReaderBase..ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00007] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Volumes.BlockVolumeReader..ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00010] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess+<>c__DisplayClass0_0.<Run>b__0 (<>f__AnonymousType92[j__TPar,j__TPar] self) [0x0023c] in :0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in <ea1db083b4fe4d4a85e24c85e3bce198>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (System.Collections.Generic.IEnumerable1[T] sources, Duplicati.Library.Snapshots.ISnapshotService snapshot, Duplicati.Library.Snapshots.UsnJournalService journalService, Duplicati.Library.Main.Operation.Backup.BackupDatabase database, Duplicati.Library.Main.Operation.Backup.BackupStatsCollector stats, Duplicati.Library.Main.Options options, Duplicati.Library.Utility.IFilter sourcefilter, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.BackupResults result, Duplicati.Library.Main.Operation.Common.ITaskReader taskreader, System.Int64 lastfilesetid) [0x00349] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00a06] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147024894,“Source”:“mscorlib”,“FileNotFound_FileName”:”/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3",“FileNotFound_FusionLog”:null},“HelpURL”:null,“StackTraceString”:" at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00ffe] in :0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“InnerExceptions”:[{“ClassName”:“System.IO.FileNotFoundException”,“Message”:“Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3"”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in :0 \n at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in :0 \n at (wrapper remoting-invoke-with-check) System.IO.FileStream…ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)\n at Duplicati.Library.Main.Volumes.VolumeReaderBase.LoadCompressor (System.String compressor, System.String file, Duplicati.Library.Main.Options options, System.IO.Stream& stream) [0x00000] in :0 \n at Duplicati.Library.Main.Volumes.VolumeReaderBase…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00007] in :0 \n at Duplicati.Library.Main.Volumes.BlockVolumeReader…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00010] in :0 \n at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess+<>c__DisplayClass0_0.b__0 (<>f__AnonymousType92[<Input>j__TPar,<Output>j__TPar] self) [0x0023c] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (System.Collections.Generic.IEnumerable1[T] sources, Duplicati.Library.Snapshots.ISnapshotService snapshot, Duplicati.Library.Snapshots.UsnJournalService journalService, Duplicati.Library.Main.Operation.Backup.BackupDatabase database, Duplicati.Library.Main.Operation.Backup.BackupStatsCollector stats, Duplicati.Library.Main.Options options, Duplicati.Library.Utility.IFilter sourcefilter, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.BackupResults result, Duplicati.Library.Main.Operation.Common.ITaskReader taskreader, System.Int64 lastfilesetid) [0x00349] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00a06] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147024894,"Source":"mscorlib","FileNotFound_FileName":"/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3","FileNotFound_FusionLog":null},{"ClassName":"System.AggregateException","Message":"One or more errors occurred.","Data":null,"InnerException":{"NativeErrorCode":10054,"ClassName":"System.Net.Sockets.SocketException","Message":"Connection reset by peer","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.Net.Sockets.Socket.EndSend (System.IAsyncResult asyncResult) [0x00012] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x00057] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType111[j__TPar] self) [0x00673] in :0 \n at Duplicati.Library.Main.Operation.Backup.BackendUploader.b__13_0 (<>f__AnonymousType111[<Input>j__TPar] self) [0x0089e] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147467259,“Source”:“mscorlib”},“HelpURL”:null,“StackTraceString”:null,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233088,“Source”:null,“InnerExceptions”:[{“NativeErrorCode”:10054,“ClassName”:“System.Net.Sockets.SocketException”,“Message”:“Connection reset by peer”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” at System.Net.Sockets.Socket.EndSend (System.IAsyncResult asyncResult) [0x00012] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x00057] in <906205e228b04a3e9ab3c0712759f5c5>:0 \n— End of stack trace from previous location where exception was thrown —\n\n at Duplicati.Library.Main.Operation.Backup.BackendUploader.b__13_0 (<>f__AnonymousType111[<Input>j__TPar] self) [0x00673] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType111[j__TPar] self) [0x0089e] in :0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func`2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147467259,“Source”:“mscorlib”}]}]}]}

16. Sept. 2019 03:04: Fatal error
{“ClassName”:“System.IO.FileNotFoundException”,“Message”:“Could not find file “/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3"”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x0019e] in :0 \n at System.IO.FileStream…ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in :0 \n at (wrapper remoting-invoke-with-check) System.IO.FileStream…ctor(string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)\n at Duplicati.Library.Main.Volumes.VolumeReaderBase.LoadCompressor (System.String compressor, System.String file, Duplicati.Library.Main.Options options, System.IO.Stream& stream) [0x00000] in :0 \n at Duplicati.Library.Main.Volumes.VolumeReaderBase…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00007] in :0 \n at Duplicati.Library.Main.Volumes.BlockVolumeReader…ctor (System.String compressor, System.String file, Duplicati.Library.Main.Options options) [0x00010] in :0 \n at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess+<>c__DisplayClass0_0.b__0 (<>f__AnonymousType92[<Input>j__TPar,<Output>j__TPar] self) [0x0023c] in <ea296dda8ab74359ac0ecb9f140cf8b1>:0 \n at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000cc] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation (System.Collections.Generic.IEnumerable`1[T] sources, Duplicati.Library.Snapshots.ISnapshotService snapshot, Duplicati.Library.Snapshots.UsnJournalService journalService, Duplicati.Library.Main.Operation.Backup.BackupDatabase database, Duplicati.Library.Main.Operation.Backup.BackupStatsCollector stats, Duplicati.Library.Main.Options options, Duplicati.Library.Utility.IFilter sourcefilter, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.BackupResults result, Duplicati.Library.Main.Operation.Common.ITaskReader taskreader, System.Int64 lastfilesetid) [0x00349] in :0 \n at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00a06] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2147024894,“Source”:“mscorlib”,“FileNotFound_FileName”:”/opt/Duplicati/tmp/dup-d006914b-9289-4daf-b9ea-2e81a397b9b3",“FileNotFound_FusionLog”:null}

Peter_Strasser · September 29, 2019, 12:17pm

By the way I didn’t figured out the problem, but I changed to another WebDav server and the problem doesn’t occur anymore.
So I guess there was a problem with the WebDav server.

ts678 · September 29, 2019, 6:49pm

I’m glad it began working. Your last (large) stack trace was too much to unravel, but it did continue to show the “Connection reset by peer” error indicating the WebDAV server closed on you. I’m not totally sure there isn’t a TLS setup problem in there somewhere because I see SecureAuthenticationFailure message.

If you are still interested enough to poke the old WebDAV server, you could try the “Test connection” button from the Destination screen. If you have an old mono version (before 4.8), it might be limited TLS 1.0, so it can cause trouble if the destination needs a higher version (e.g. TLS 1.1 or TLS 1.2). You can also have a certificate issue. Regardless I’m glad whatever it was is solved. BTW, what I hoped for on logging was like

2019-01-18 11:25:27 -05 - [Retry-Duplicati.Library.Main.BackendManager-RetryPut]: Operation Put with file duplicati-b1cce59315ad34bcd83896983e04028dd.dblock.zip.aes attempt 1 of 5 failed with message: 503 - service_unavailable: c001_v0001113_t0024 is too busy
System.Exception: 503 - service_unavailable: c001_v0001113_t0024 is too busy
   at Duplicati.Library.Backend.Backblaze.B2AuthHelper.AttemptParseAndThrowException(Exception ex)
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.Backend.Backblaze.B2.Put(String remotename, Stream stream)
   at Duplicati.Library.Main.BackendManager.DoPut(FileEntryItem item)
   at Duplicati.Library.Main.BackendManager.ThreadRun()
2019-01-18 11:25:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Retrying: duplicati-b1cce59315ad34bcd83896983e04028dd.dblock.zip.aes (50.07 MB)
2019-01-18 11:25:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-b1cce59315ad34bcd83896983e04028dd.dblock.zip.aes (50.07 MB)
2019-01-18 11:25:28 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Rename: duplicati-b76056da9bbad446aa3f42a811bd52a07.dblock.zip.aes (50.07 MB)
2019-01-18 11:25:28 -05 - [Information-Duplicati.Library.Main.BackendManager-RenameRemoteTargetFile]: Renaming "duplicati-b1cce59315ad34bcd83896983e04028dd.dblock.zip.aes" to "duplicati-b76056da9bbad446aa3f42a811bd52a07.dblock.zip.aes"
2019-01-18 11:25:38 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b76056da9bbad446aa3f42a811bd52a07.dblock.zip.aes (50.07 MB)
2019-01-18 11:27:55 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b76056da9bbad446aa3f42a811bd52a07.dblock.zip.aes (50.07 MB)

which shows a more readable temporary failure, followed by a rename of the file (avoids an issue, but I might have just found a bug it causes in the index file), then a retry. I don’t know if your earlier error was solid when it hit (and exhausted all 5 of the default retries) or if it was more intermittent. Glad it’s gone…