Backup to WeDAV / Nextcloud never completes

hell · April 7, 2021, 8:19am

Hello there,

i would like to backup a few hundert Gbytes to my Nextcloud. Basically ist works. After a while the jobs breaks for some reason. Duplicati say something like “at least one error happend”. If i click on show a do not some userful report.

I tried to modify
synchronous-concurrent-upload-limit 1
concurrency-max-threads 1
debug-retry-errors x
http-operation-timeout 1 min
http-readwrite-timeout 1 min
log-file-log-level Info
retry-delay 21s
zip-compression-level 0

but there’s no real improvement that i can see.

Maybe someone has an idea what to to?

regards Karsten

ts678 · April 9, 2021, 12:24am

Welcome to the forum @hell

Backups that error out often have no log in the usual spot. Try server log at About → Show log → Stored, however a better approach might be About → Show log → Live → Retry to see if there are upload errors, which seems to be your focus area for settings changes. If you see an error, you can try clicking that line.

hell · April 9, 2021, 6:38am

Hi ts678,
that was a good hint! i didn’t knew, that the logs can be find there.

Setting to verbose gave me this error as resul. Any idea, why the file dup-ce52f079-e628-4d30-bf83-08b028d1b435 suddently cannot be found?

{“ClassName”:“System.IO.FileNotFoundException”,“Message”:“Die Datei “C:\Users\hell\AppData\Local\Temp\dup-ce52f079-e628-4d30-bf83-08b028d1b435” konnte nicht gefunden werden.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)\r\n bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)\r\n bei System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)\r\n bei Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei CoCoL.AutomationExtensions.d__10`1.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei Duplicati.Library.Main.Operation.BackupHandler.d__13.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nWinIOError\nmscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089\nSystem.IO.__Error\nVoid WinIOError(Int32, System.String)”,“HResult”:-2147024894,“Source”:“mscorlib”,“WatsonBuckets”:null,“FileNotFound_FileName”:“C:\Users\hell\AppData\Local\Temp\dup-ce52f079-e628-4d30-bf83-08b028d1b435”,“FileNotFound_FusionLog”:null}

ts678 · April 9, 2021, 12:07pm

“Suddenly” without context is hard. Can you show some logs leading up to that? Verbose level might do, however Information level might also be enough. You also have a status bar near screen top with status.

Could not find the file…. /tmp is a similar problem undergoing troubleshooting right now. Historically, this problem seems to happen near the end of the backup. Network errors are also suspected. Need logs…

EDIT 1:

Connection dropout appears to corrupt database was one where a much larger backup than yours might overload something in Nextcloud and/or Duplicati on the list done at end of backup. Need logs to see…

EDIT 2:

The above issue gets into the possibility of doing unusually heavy logging efforts, if in fact the list failed.

Lacking logs, and because it sounds like you are just getting going (having no existing backups), it may be worth testing the latest Canary build v2.0.5.114-2.0.5.114_canary_2021-03-10 to see if its changes fix this. Sometimes Canary is very high risk, but this one should be very similar to the next Beta, hoped for soon…

hell · April 9, 2021, 1:47pm

Hi ts678,

maybe you can have a look at the logs at first and see then, what might help…

9 Apr 2021 15:34: Fatal error
9 Apr 2021 15:34: Backend event: Put - Failed: duplicati-bdf7fe1b7c5fa476b94fc3d952457a04d.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:30: Backend event: Put - Started: duplicati-bdf7fe1b7c5fa476b94fc3d952457a04d.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:30: Renaming “duplicati-bff2ff903d7c64690aeeeb9eafa173bca.dblock.zip.aes” to “duplicati-bdf7fe1b7c5fa476b94fc3d952457a04d.dblock.zip.aes”
9 Apr 2021 15:30: Backend event: Put - Rename: duplicati-bdf7fe1b7c5fa476b94fc3d952457a04d.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:30: Backend event: Put - Rename: duplicati-bff2ff903d7c64690aeeeb9eafa173bca.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:30: Backend event: Put - Retrying: duplicati-bff2ff903d7c64690aeeeb9eafa173bca.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:25: Backend event: Put - Started: duplicati-bff2ff903d7c64690aeeeb9eafa173bca.dblock.zip.aes (1,023.91 MB)
9 Apr 2021 15:25: Renaming “duplicati-b20caf538352a453fa57fa1c0e5820104.dblock.zip.aes” to “duplicati-bff2ff903d7c64690aeeeb9eafa173bca.dblock.zip.aes”
[…]

I opened the fatal error entry and it contains this:

{“ClassName”:“System.IO.FileNotFoundException”,“Message”:“Die Datei “C:\Users\hell\AppData\Local\Temp\dup-b4f61154-9948-448d-bf2c-85fdf133408c” konnte nicht gefunden werden.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" bei System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)\r\n bei System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)\r\n bei System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)\r\n bei Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei CoCoL.AutomationExtensions.d__10`1.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei Duplicati.Library.Main.Operation.BackupHandler.d__13.MoveNext()\r\n— Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde —\r\n bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n bei Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nWinIOError\nmscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089\nSystem.IO.__Error\nVoid WinIOError(Int32, System.String)”,“HResult”:-2147024894,“Source”:“mscorlib”,“WatsonBuckets”:null,“FileNotFound_FileName”:“C:\Users\hell\AppData\Local\Temp\dup-b4f61154-9948-448d-bf2c-85fdf133408c”,“FileNotFound_FusionLog”:null}

ts678 · April 9, 2021, 2:16pm

That added a tiny amount, but didn’t go far enough back. The missing file isn’t news, but this gave a better view of how it seemingly interferes with uploading. I guess if the file to upload is missing, upload breaks… What’s needed is a look at what was done earlier, especially if anything went into retries before final error.

It should be possible to scroll the GUI to an earlier point. If you wish, you can also set up a log file such as log-file=<path> at log-file-log-level=retry for starters. If need be, there are more detailed logs, e.g. verbose.

hell · April 9, 2021, 6:31pm

Hi ts678,

i sent you the logs directly.

To complete the information…

duplicti is running on a Win10 machine
i have addtional 3 more jobs configured
- here we have NO issues, they are running perfectly
- difference is, its only between Windows shares in my LAN

ts678 · April 9, 2021, 7:14pm

Does that mean the backup destination? So the failures are where Nextcloud is the destination?
Nextcloud in response to its very first upload (put) request from Duplicati gives an internal error:

2021-04-09 17:52:11 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-bdd2e139aa2744116915d265416d85209.dblock.zip.aes (1,023.92 MB)
2021-04-09 17:56:24 +02 - [Retry-Duplicati.Library.Main.Operation.Backup.BackendUploader-RetryPut]: Operation Put with file duplicati-bdd2e139aa2744116915d265416d85209.dblock.zip.aes attempt 1 of 9 failed with message: Der Remoteserver hat einen Fehler zurückgegeben: (500) Interner Serverfehler.
System.Net.WebException: Der Remoteserver hat einen Fehler zurückgegeben: (500) Interner Serverfehler.
   bei Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   bei Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   bei Duplicati.Library.Backend.WEBDAV.<PutAsync>d__36.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
   bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bei Duplicati.Library.Main.Operation.Backup.BackendUploader.<DoPut>d__21.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
   bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   bei System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   bei Duplicati.Library.Main.Operation.Backup.BackendUploader.<>c__DisplayClass17_0.<<UploadFileAsync>b__0>d.MoveNext()
--- Ende der Stapelüberwachung vom vorhergehenden Ort, an dem die Ausnahme ausgelöst wurde ---
   bei System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   bei Duplicati.Library.Main.Operation.Backup.BackendUploader.<DoWithRetry>d__18.MoveNext()

Retries continue until exhausted, at which point the FileNotFoundException on Temp file occurs again.
I don’t have any way to force the exact error, but maybe I (or a developer) could force a similar error…

Do you have any idea why your Nextcloud is giving an internal error? It’s also taking it a lot of minutes.
Is this local or over the Internet with some speed limits? I see you have a somewhat oversized remote volume size of what looks like 1024 MB (up from 50). I wonder if the uploads timed out, causing error?

If you think it might be timing, you can lower the remote volume size back to default to see if that helps.

hell · April 13, 2021, 3:57am

At first: it was a fault at the Nextcloud/Provider!

ts678, you mentioned an internal error at the Nextcloud. So i decided to run a minor update of Nextcloud, which is hostet at an Nextcloud Provider.This failed and made the Nextcloud inaccessable. I raised a ticket at the provider and they repaired it. That did’nt tell what they repaired, but now Duplicati is running as it should.

So i thank you for supporting me!