FileNotFoundException (temp file not found during backup)

Daniel_Eriksson · October 12, 2017, 10:43am

In the middle of a backup to Google Drive the following exception was thrown. Is this a known issue?

System.IO.FileNotFoundException: Could not find file 'C:\Users\Daniel\AppData\Local\Temp\dup-fa76e609-d5cd-4b1f-bf66-613cf023453b'.
File name: 'C:\Users\Daniel\AppData\Local\Temp\dup-fa76e609-d5cd-4b1f-bf66-613cf023453b'
   at Duplicati.Library.Main.Operation.BackupHandler.HandleFilesystemEntry(ISnapshotService snapshot, BackendManager backend, String path, FileAttributes attributes)
   at Duplicati.Library.Main.Operation.BackupHandler.RunMainOperation(ISnapshotService snapshot, BackendManager backend)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

The exception cause the backup job to fail halfway through. The job had already uploaded ~50 GB to Google Drive. If I restart the backup it will start from the beginning and leave a lot of orphaned files on Google Drive. Is there a way to restart the job from where it crashed, or at least a way to prune the orphaned files before I restart from scratch?

kenkendk · October 12, 2017, 1:31pm

No, I am not aware of this being an issue. Are you perhaps running something like CCleaner or similar that could wipe the temp files?

That should not be a problem. Just run the backup again, and it will figure out that the data is already uploaded and not upload it again.

Daniel_Eriksson · October 12, 2017, 1:58pm

I do have CCleaner installed but it does not run in the background. To the best of my knowledge there is nothing on my system that would delete temp-files automatically. If it happens again I’ll dig deeper.

Daniel_Eriksson · October 12, 2017, 2:21pm

I was wrong. It tuns out that I do have something that automatically deletes temporary files. I just switched from “Norton Internet Security” to “Norton Security Deluxe”, and apparently one of the new features is automatic cleanup of temp files.

If the cleanup algorithm is stupid enough to delete files that are just a few minutes old then this is a really bad implementation. I assume the Duplicati file was pretty big, and as such a prime target for cleanup, but Norton should not delete files that are just a few minutes old no matter how big they are.

kenkendk · October 12, 2017, 8:09pm

I agree, hopefully you can disable the feature. Otherwise, use the --tempdir advanced option to set it to someplace that is not auto-cleaned.

JonMikelV · October 12, 2017, 10:57pm

I’m not exactly sure what temp file was involved but (somebody please correct me if I’m wrong) I would expect it to be not much larger than your dblock size - are you using a custom size (the default is 50MB) that might have made it a cleanup target?

Do we know if this happens often enough to handle this as a warning with a restart of that file and only fail after X number of times the temp file disappears?

kenkendk · October 16, 2017, 1:29pm

Not sure, but I would rather fix it by keeping the file “in-memory”, as it is likely to not be much of a problem (say 50-200mb file, and memory is 4GB+ on new laptops).

But it is a bit of a corner case, as the file is temporary (and thus is written at the right place) but it is also used to “communicate” internally, in that it is created and passed to another method. It would be possible to keep the file open, which would prevent a cleaner program from deleting it.

kees-z · October 16, 2017, 1:40pm

For me this looks like the best way to handle this. Better to avoid problems than try dealing with them.

JonMikelV · October 16, 2017, 5:25pm

I agree with two caveats:

I’ve seen a number of people posting about running Duplicati on NAS devices / in Docker containers, so memory may not be as plentiful in such environments. Then again, cleanup tasks wouldn’t likely be running there either, so maybe a parameter to en/disable temp-files-in-memory would be good
I’ve seen some people talking about 500M - 1G dblock sizes, so again maybe a parameter to en/disable temp-files-in-memory would be good

kenkendk · October 16, 2017, 7:24pm

Yes. The idea is to auto-enable in-memory storage if the block size is 10% of the system memory (or less).

rick55 · June 25, 2020, 4:59pm

Need to raise this topic again. Running duplicati 2.0.5.1 on Windows 7 (hardware is pretty old which is why I haven’t gone to 10 yet). Machine has 32 GB of memory and 50G of free space on the C (system & temp drive, SSD). I’ve also tried to use the tempdir parameter to point to an empty 1 TB SSD with the same issue. Dblock size is the default 50MB, but I’m trying to back up a directory that has almost 300,000 files and is 1.8 TB in size before compression. Apologies if this issue has been raised recently and I’ve missed it.

Thanks
Rick

ts678 · June 26, 2020, 9:38pm

Please post exact stack trace. The one in original post hasn’t been reported since 2017, but similar have.
Specifically, file here is missing in HandleFileSystemEntry. Below one fails in SpillCollectorProcess work:

Another backup failed due to error Cannot find file

I have a theory that something else goes wrong first, so more logs may be needed. But first, which case?

rick55 · June 27, 2020, 5:44am

I have 2 different backups on the same system failing in apparently the same manor.

Here is the first stack trace:

Result

failed

Errors

Duplicati Backup report for zz.s.backups

Failed: One or more errors occurred.
Details: System.AggregateException: One or more errors occurred. —> System.AggregateException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’. —> System.IO.FileNotFoundException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CoCoL.AutomationExtensions.d__101.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunMainOperation>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of inner exception stack trace --- at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of inner exception stack trace --- at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)
—> (Inner Exception #0) System.AggregateException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’. —> System.IO.FileNotFoundException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CoCoL.AutomationExtensions.d__101.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunMainOperation>d__13.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of inner exception stack trace --- at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() ---> (Inner Exception #0) System.IO.FileNotFoundException: Could not find file 'C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266'. File name: 'C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266' at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<<Run>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at CoCoL.AutomationExtensions.<RunTask>d__101.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.BackupHandler.d__13.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()<—

—> (Inner Exception #1) System.AggregateException: One or more errors occurred. —> System.Net.WebException: The remote server returned an error: (507) Insufficient Storage.
at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
at Duplicati.Library.Backend.WEBDAV.d__36.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__21.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.<>c__DisplayClass17_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__18.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__17.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.d__15.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.Backup.BackendUploader.<b__13_0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Duplicati.Library.Main.Operation.Backup.BackendUploader.<b__13_0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CoCoL.AutomationExtensions.d__101.MoveNext() --- End of inner exception stack trace --- ---> (Inner Exception #0) System.Net.WebException: The remote server returned an error: (507) Insufficient Storage. at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream() at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse() at Duplicati.Library.Backend.WEBDAV.<PutAsync>d__36.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<DoPut>d__21.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<>c__DisplayClass17_0.<<UploadFileAsync>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<DoWithRetry>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Duplicati.Library.Main.Operation.Backup.BackendUploader.<DoWithRetry>d__18.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<UploadFileAsync>d__17.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<UploadBlockAndIndexAsync>d__15.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Duplicati.Library.Main.Operation.Backup.BackendUploader.<<Run>b__13_0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at Duplicati.Library.Main.Operation.Backup.BackendUploader.<<Run>b__13_0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at CoCoL.AutomationExtensions.<RunTask>d__101.MoveNext()<—
<—
<—

Log data:
2020-06-24 19:45:10 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\Downloads\facemoods.exe.917.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\Downloads\facemoods.exe.917.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:02:12 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2016.older\080216\Downloads\facemoods.exe.917.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2016.older\080216\Downloads\facemoods.exe.917.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:08:12 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2016.older\010116\monica’s.sync\123115\Downloads\facemoods.exe.917.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2016.older\010116\monica’s.sync\123115\Downloads\facemoods.exe.917.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:19:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2015.older\Downloads\Facemoods.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2015.older\Downloads\Facemoods.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:19:05 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2015.older\Downloads\facemoods.exe.118288.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2015.older\Downloads\facemoods.exe.118288.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:19:06 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2015.older\Downloads\facemoods.exe.917.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2015.older\Downloads\facemoods.exe.917.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:20:11 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2015.older\Downloads\YourFile_downloader.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2015.older\Downloads\YourFile_downloader.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:20:11 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\2015.older\Downloads{REQUEST}_downloader.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\2015.older\Downloads{REQUEST}_downloader.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:29:30 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\123115\Downloads\Facemoods.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\123115\Downloads\Facemoods.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:29:30 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\123115\Downloads\facemoods.exe.917.gzquar
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\123115\Downloads\facemoods.exe.917.gzquar’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:30:27 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\123115\Downloads\YourFile_downloader.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\123115\Downloads\YourFile_downloader.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-24 20:30:28 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: S:\S-drive\backups\Monica\123115\Downloads{REQUEST}_downloader.exe
System.UnauthorizedAccessException: Access to the path ‘S:\S-drive\backups\Monica\123115\Downloads{REQUEST}_downloader.exe’ is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Common.IO.SystemIOWindows.<>c.b__29_0(String p)
at Duplicati.Library.Common.IO.SystemIOWindows.PathTooLongFuncWrapper[T](Func2 nativeIOFunc, Func2 alternativeIOFunc, String path, Boolean prefixWithUnc)
at Duplicati.Library.Common.IO.SystemIOWindows.FileOpenRead(String path)
at Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.<>c__DisplayClass1_0.<b__0>d.MoveNext()
2020-06-25 01:48:53 -07 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
System.IO.FileNotFoundException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’.
File name: ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at CoCoL.AutomationExtensions.d__10`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.BackupHandler.d__13.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()

Time of report

2020-06-25 01:49:12

Last change

2020-06-25 01:49:12

ts678 · June 27, 2020, 10:45am

This is the SpillCollectorProcess, so maybe belongs in the other (more active) thread, however let’s see what your other stack trace looks like (if you were planning to post it) before maybe moving debug efforts. Feel free to go over to the other one to see what the discussion has been. Probably some applies here…

rick55:

Details: System.AggregateException: One or more errors occurred. —> System.AggregateException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’. —> System.IO.FileNotFoundException: Could not find file ‘C:\Windows\TEMP\dup-c221087b-9693-4057-aecb-5940467ee266’.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
at Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.

I’d note that the original post in the other never got specific, but a later participant posted this stack trace:

Another backup failed due to error Cannot find file

Details: System.AggregateException: Er zijn één of meer fouten opgetreden. —> System.AggregateException: Kan bestand C:\WINDOWS\TEMP\dup-c6adb50d-67f2-456d-aae5-b059e229357a niet vinden. —> System.IO.FileNotFoundException: Kan bestand C:\WINDOWS\TEMP\dup-c6adb50d-67f2-456d-aae5-b059e229357a niet vinden.
bij System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
bij System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
bij System.IO.FileStream…ctor(String path, FileMode mode, FileAccess access, FileShare share)
bij Duplicati.Library.Main.Operation.Backup.SpillCollectorProcess.<>c__DisplayClass0_0.<b__0>d.MoveNext()

For yours, the “something else goes wrong first” might be below. If reproducible, you can watch About → Show log → Live → Retry or set up a –log-file at –log-file-log-level=retry to get a better look at the server:

Any idea what that’s about? Does the WebDAV server have any tools or logging that might show its view?
Although this is simply comparing bits of errors from various people, original author of other topic had this:

but didn’t link to the other thread. Maybe you can go see if you can find it, to add to the evidence collecting.

rick55 · June 28, 2020, 9:45am

My problem was fixed by checking my quota in nextcloud, which apparently allows no overage space. This was the case for both stack traces I submitted. I don’t know why both stack traces didn’t indicate where the space was insufficient. That would have several a lot of headaches.

Thanks, as always ts, for jumping in with both feet.

Rick

ts678 · June 28, 2020, 6:18pm

is the first. Where’s the second? BTW a stack trace is usually the result of something unexpected happening, is not a “designed” part of the program or UI, and is not even formatted by the program.

The designed UI for reporting insufficient space is warnings and maybe errors based on space use, however WebDAV doesn’t report space use. All it does it error when you’re over. You saw this error:

11.5. 507 Insufficient Storage

   The 507 (Insufficient Storage) status code means the method could not
   be performed on the resource because the server is unable to store
   the representation needed to successfully complete the request.  This
   condition is considered to be temporary.  If the request that
   received this status code was the result of a user action, the
   request MUST NOT be repeated until it is requested by a separate user

If the error is still there (or you’re willing to cause it again), it would still be good to get some logs to see exactly how the destination error (presumably an upload, but Duplicati has many upload phases) hurts. Potentially that will let the problem be understood then fixed, so that other people aren’t hitting it as well.

quota-size option can be set up by you if you know your storage limit. This will give a warning when close:

Backend quota is close to being exceeded

EDIT:

RFC 4331 – Quota and Size Properties for Distributed Authoring and Versioning (DAV) Collections
potentially offers a way for Duplicati to get the quota information it needs from WebDAV servers that do it.
Until such enhancement, WebDAV backend isn’t an IQuotaEnabledBackend, so quota–size should work:

–quota-size
This value can be used to set a known upper limit on the amount of space a backend has. If the backend reports the size itself, this value is ignored.

rick55 · June 28, 2020, 8:49pm

If I decide to try canary with the new database, it would be on these backups (since I have other copies). I will consider inducing the error then. What logging should I turn in? I’ve read so many articles about log levels and some things have been deprecated but not removed as an option, etc.

Thanks for the ‘quota-size’ option. I’ll use this to help keep tabs on my backups, as nextcloud has no settings between 250G and unlimited.

And all your other help

Rick

ts678 · June 28, 2020, 11:04pm

For your decision, note that Canary has the latest bug fixes and features, but also may have new bugs, so using it for valuable backups is a judgment call. Also, newer Duplicati can have database version updates, meaning going back to the old version can be awkward (one way is by the backup file that’s made for you).

The error is probably not version-dependent. That’s not known certainly, but I haven’t heard of any such fix.

–log-file at –log-file-log-level=retry is probably a good place to start. Live log is easier to run, harder to post. I’m not sure how much activity is before error, but even a large log can be turned into a .zip file and posted. Small amounts can be pasted into the forum with three backticks (```) above and below to stop formatting.

Thanks for any light you can give on what goes on during a failed run. I hope heavy collecting isn’t needed. It’s a shame the temporary file names are all alike, so it’s hard to know what that missing file was used for.