Failed while executing "Backup" with id: 2

frankieandshadow · October 11, 2017, 3:45pm

Hi,

I’ve been trying a number of times to get a complete first backup, and not yet succeeded. My most recent attempt ran for some 7.5 hours and then got an error, and then kept repeatedly generating errors like this until it gave up about 3 hours later:

System.Exception: Unexpected number of remote volumes detected: 0!
   at Duplicati.Library.Main.Database.LocalDatabase.UpdateRemoteVolume(String name, RemoteVolumeState state, Int64 size, String hash, Boolean suppressCleanup, TimeSpan deleteGraceTime, IDbTransaction transaction)
   at Duplicati.Library.Main.Database.LocalDatabase.UpdateRemoteVolume(String name, RemoteVolumeState state, Int64 size, String hash, Boolean suppressCleanup, IDbTransaction transaction)
   at Duplicati.Library.Main.Operation.BackupHandler.FinalizeRemoteVolumes(BackendManager backend)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

This is to a Raspberry Pi running Raspbian Jessie with a large powered external disk, sending over SSH, trying to backup around 400GB. It looks like it got about 20% of the way through. The remote server was still operating, and continued to access the same disk both before, during and after the backups. There is nothing in the Pi’s error logs to indicate a network problem or disk problem.

I can see the files on the server start at 16:38 yesterday, and continue until 23:34. Then thousands of error messages then continue until 02:32, with…

Rollback error: unknown error
No transaction is active on this connection
System.Data.SQLite.SQLiteException (0x80004005): unknown error
No transaction is active on this connection
   at System.Data.SQLite.SQLiteTransactionBase.IsValid(Boolean throwError)
   at System.Data.SQLite.SQLiteTransactionBase.Rollback()
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)

and (for example)

System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Main.Volumes.BlockVolumeWriter.AddBlock(String hash, Byte[] data, Int32 offset, Int32 size, CompressionHint hint)
   at Duplicati.Library.Main.Operation.BackupHandler.AddBlockToOutput(BackendManager backend, String key, Byte[] data, Int32 offset, Int32 len, CompressionHint hint, Boolean isBlocklistData)
   at Duplicati.Library.Main.Operation.BackupHandler.ProcessStream(Stream stream, CompressionHint hint, BackendManager backend, FileBackedStringList blocklisthashes, FileBackedStringList hashcollector, Boolean skipfilehash)
   at Duplicati.Library.Main.Operation.BackupHandler.HandleFilesystemEntry(ISnapshotService snapshot, BackendManager backend, String path, FileAttributes attributes)

at which point it stops.

kenkendk · October 12, 2017, 1:19pm

This means that the files were uploaded, but when Duplicati attempts to list the remote folder it returns “zero files found”. Can you check that the remote folder has files? Also, there is a Duplicati.CommandLine.BackendTool.exe that can be used to query the destination and “see things like Duplicati does”, maybe this can reveal the cause?

These happen after something else breaks, and the backup really should stop, but due to a poor choice in implementation it does not. I am working on a rewrite that fixes this.

frankieandshadow · October 13, 2017, 10:57am

Thanks for the reply.

Yes there are files on the remote server.

Before you replied, I started it again, and it appeared to run to completion that time, with a few errors about locked files, and ones which I assume are transient files that had vanished between preparation and attempting to upload - mostly to do with my existing backups which are Crashplan (which is going away before long, hence my searching for a reliable alternative).

However, since then I’ve run it again (I’m trying to set up a schedule), and it failed again in the same way. And I also tried setting it up to a local USB disk instead of ssh, and that also gave the same error at the same point.

I’m also now seeing lots of errors like this:

Failed to process path: [pathname-redacted].png
System.NullReferenceException: Object reference not set to an instance of an object.
   at Duplicati.Library.Main.Volumes.BlockVolumeWriter.AddBlock(String hash, Byte[] data, Int32 offset, Int32 size, CompressionHint hint)
   at Duplicati.Library.Main.Operation.BackupHandler.AddBlockToOutput(BackendManager backend, String key, Byte[] data, Int32 offset, Int32 len, CompressionHint hint, Boolean isBlocklistData)
   at Duplicati.Library.Main.Operation.BackupHandler.ProcessStream(Stream stream, CompressionHint hint, BackendManager backend, FileBackedStringList blocklisthashes, FileBackedStringList hashcollector, Boolean skipfilehash)
   at Duplicati.Library.Main.Operation.BackupHandler.AddMetadataToOutput(BackendManager backend, IMetahash meta)
   at Duplicati.Library.Main.Operation.BackupHandler.HandleFilesystemEntry(ISnapshotService snapshot, BackendManager backend, String path, FileAttributes attributes)

I tried running the backendtool you mentioned and got a long-ish report with lots of errors appended below.

As there are always locked files and transient files, every run ends in failure. It would be really helpful to distinguish between errors that can reasonably happen (they aren’t really errors, just changes in circumstance while it is backing up, which after all takes many hours), like those, and ones which are catastrophic. There’s so many different error messages everywhere, at the moment I can’t have any confidence as to whether it is working fully or not. I can’t see the wood for the trees! Here’s another one:

Failed to process metadata for "C:\Users\DefaultAppPool\", storing empty metadata
System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.
   at System.Security.AccessControl.Win32.GetSecurityInfo(ResourceType resourceType, String name, SafeHandle handle, AccessControlSections accessControlSections, RawSecurityDescriptor& resultSd)
   at System.Security.AccessControl.NativeObjectSecurity.CreateInternal(ResourceType resourceType, Boolean isContainer, String name, SafeHandle handle, AccessControlSections includeSections, Boolean createByName, ExceptionFromErrorCode exceptionFromErrorCode, Object exceptionContext)
   at System.Security.AccessControl.FileSystemSecurity..ctor(Boolean isContainer, String name, AccessControlSections includeSections, Boolean isDirectory)
   at System.Security.AccessControl.DirectorySecurity..ctor(String name, AccessControlSections includeSections)
   at Duplicati.Library.Snapshots.SystemIOWindows.GetAccessControlDir(String path)
   at Duplicati.Library.Snapshots.SystemIOWindows.GetMetadata(String path, Boolean isSymlink, Boolean followSymlink)
   at Duplicati.Library.Main.Operation.BackupHandler.GenerateMetadata(ISnapshotService snapshot, String path, FileAttributes attributes)

and

No hash or size recorded for duplicati-b405e31f23dbc493399373523f329a994.dblock.zip.aes, performing full verification

Here’s the BackendTool output. I have no idea how to interpret this:
(The forum won’t let me upload all of this, so it’s truncated after a few entries)

Starting run no 0

Generating file 0 (38.90 MB)

Generating file 1 (49.74 MB)

Generating file 2 (5.36 MB)

Generating file 3 (37.98 MB)

Generating file 4 (17.61 MB)

Generating file 5 (2.81 MB)

Generating file 6 (13.80 MB)

Generating file 7 (16.93 MB)

Generating file 8 (35.09 MB)

Generating file 9 (11.96 MB)

Uploading wrong files ...

Generating file 10 (1.83 KB)

Uploading file 0, 1.83 KB ... Failed to upload file 0, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: r6AC6aYRUUNulnRkB

Uploading file 0, 1.83 KB ... Failed to upload file 0, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: r6AC6aYRUUNulnRkB

Uploading file 9, 1.83 KB ... Failed to upload file 9, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: aVT&om(#uCqu=EzB!eR

Uploading files ...

Uploading file 0, 38.90 MB ... Failed to upload file 0, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: r6AC6aYRUUNulnRkB

Uploading file 1, 49.74 MB ... Failed to upload file 1, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: 5KZ)5Xv

Uploading file 2, 5.36 MB ... Failed to upload file 2, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: w6BAq)5@aYXh@n1DjMyf'kX@MhMj4RROa-cc#mKBHTuEISYoU#m6Ai!lhF,iCVqHh

Uploading file 3, 37.98 MB ... Failed to upload file 3, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: m3c!(Z''2x=z!RkXMwzwM7MlgLAf(xqYnZO-$(Ez1'h,-x_Y8_M$rGsEq)(Dy

Uploading file 4, 17.61 MB ... Failed to upload file 4, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: FXVAArYhv(hGxBnRWkf gb

Uploading file 5, 2.81 MB ... Failed to upload file 5, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: ksnwIx_Yo5AtXi9HdO+KRtl3R8tJWTaTTWB2),,PVPrvigc252Ui9Tm(hJ4Y

Uploading file 6, 13.80 MB ... Failed to upload file 6, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: %3j_$ZtRLbBbr#bDu@8LV7R,Qxl%+9zbK#jz9v

Uploading file 7, 16.93 MB ... Failed to upload file 7, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: IbPntj!U$_Ac2j

Uploading file 8, 35.09 MB ... Failed to upload file 8, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: + r4&rVLC7@4V3'R7%)zHV_mX@B8Ajdfuj8VEqa$LL7C=Ji

Uploading file 9, 11.96 MB ... Failed to upload file 9, error message: Renci.SshNet.Common.SftpPermissionDeniedException: Permission denied

   at Renci.SshNet.Sftp.SftpSession.RequestOpen(String path, Flags flags, Boolean nullOnError)

   at Renci.SshNet.SftpClient.InternalUploadFile(Stream input, String path, Flags flags, SftpUploadAsyncResult asyncResult, Action`1 uploadCallback)

   at Duplicati.Library.Backend.SSHv2.Put(String remotename, Stream stream)

   at Duplicati.CommandLine.BackendTester.Program.Uploadfile(String localfilename, Int32 i, String remotefilename, IBackend backend, Boolean disableStreaming), remote name: aVT&om(#uCqu=EzB!eR

Verifying file list ...

*** File with name r6AC6aYRUUNulnRkB was uploaded but not found afterwards

*** File with name 5KZ)5Xv was uploaded but not found afterwards

*** File with name w6BAq)5@aYXh@n1DjMyf'kX@MhMj4RROa-cc#mKBHTuEISYoU#m6Ai!lhF,iCVqHh was uploaded but not found afterwards

*** File with name m3c!(Z''2x=z!RkXMwzwM7MlgLAf(xqYnZO-$(Ez1'h,-x_Y8_M$rGsEq)(Dy was uploaded but not found afterwards

*** File with name FXVAArYhv(hGxBnRWkf gb was uploaded but not found afterwards

*** File with name ksnwIx_Yo5AtXi9HdO+KRtl3R8tJWTaTTWB2),,PVPrvigc252Ui9Tm(hJ4Y was uploaded but not found afterwards

*** File with name %3j_$ZtRLbBbr#bDu@8LV7R,Qxl%+9zbK#jz9v was uploaded but not found afterwards

*** File with name IbPntj!U$_Ac2j was uploaded but not found afterwards

JonMikelV · October 13, 2017, 4:18pm

Would I be correct in guessing you’re running the tray icon (not a service) on Windows and trying to back up the C:\Users folder? I ask because that’s the same error I got when using Duplicati that way. If that IS what you’re doing, then that particular issue is likely due to your user account not having permissions to see into the account folders for other users.

I have gotten around that specific issue in a few different ways - hopefully one of them will work for you:

Try to browse to the offending folder. If prompted that you don’t have access but can “click here to get access”, do so. After that my backup run without that error. Note that technically doing this circumvents individual user privacy as you would then have access to all their files (which you need so you can do the backup).
Run Duplicati as a service. The default service account has access to just about all folders on the machine. Again, doing this made my backups run without that error. While this does also circumvents individual user privacy, it’s only through the restoration of backups that you would be able to look at another users files.
Run Duplicati tray for each user and only back up that user’s C:\Users\xxx folder. This has side effects:
a. Pro: not processing files that likely haven’t changed (if the user didn’t log on, their files likely didn’t change)
b. Pro: each user’s backup has it’s own encryption key so security is maintained all they way through restores
c. Con: de-duplication only happens at the backup level, so if you have 5 users each running a backup they’ll only de-dupe within their own files

The rest of the errors I’m not sure sure about, but perhaps removing one type of problem from your list will help thin the trees a bit.

frankieandshadow · October 13, 2017, 4:59pm

I just installed with whatever the default was, so presumably yes.

Rather than reinstall just at the moment, I’ve just changed the path to include only C:\Users[me] rather than all of C:\Users. I’m the only user of this machine, so that’s not an issue.

JonMikelV · October 13, 2017, 5:27pm

That works. I know my users folder on my single-user Windows machine has all sorts of other accounts that I thank came in with other apps or something.

Oh, and running as a service doesn’t require a re-install - just a command line call to enable the service. But if the single user run works for you, stick with that.

For the “permission denied” errors on the upload, it sounds like there may be an issue on the SSH side of things. Could the destination disk be full or perhaps have a disk space quota that you’ve hit? It’s possible, though unlikely, that you’ve managed to hit a max-files-per-folder OS limit…

You might also consider checking things with a small backup run going to a different folder of the same SSH destination (use the same user login as well).

frankieandshadow · October 13, 2017, 5:47pm

Actually, I’m going to start over. I’ve got hopelessly confused as to what happened when, so I’m going to start with a clean install, all files deleted, and start with a local disk before ssh.

However, the remote disk is huge, definitely not full, and the local disk also got the same error on its run.

More tomorrow, when the new install has had a chance to run…

JonMikelV · October 13, 2017, 5:53pm

Good call on testing to local disk - that removes a lot of variables.

frankieandshadow · October 14, 2017, 12:07pm

So, to the local USB disk it failed with a number of errors, finally giving up with a purported “disk I/O error”, which I don’t for a moment believe is the real cause. The disk is not full, and is working fine, both before and after. (I am just a little suspicious that it failed exactly at midnight, but probably just coincidence. Here is the complete remote log below.

If I go to restore (which, by the way, takes a completely unusably long time to respond to every action), it shows one of the three disks it was backing up to be there completely (C:\Users\User), another (D: the largest one) to be partly done, and a third (E:) not to be there at all. So it presumably failed when doing D.

This is, I think, the fifth time I have tried to do a backup, and have so far failed to get anywhere near, with multiple different errors every time. This isn’t encouraging for backup software, which has to be super-reliable.

14 Oct 2017 00:00: Failed while executing "Backup" with id: 1
System.Data.SQLite.SQLiteException (0x80004005): disk I/O error
disk I/O error
   at System.Data.SQLite.SQLite3.Reset(SQLiteStatement stmt)
   at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
   at System.Data.SQLite.SQLiteDataReader.NextResult()
   at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
   at System.Data.SQLite.SQLiteCommand.ExecuteReader(CommandBehavior behavior)
   at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(CommandBehavior behavior)
   at Duplicati.Library.Main.BasicResults.LogDbMessage(String type, String message, Exception ex)
   at Duplicati.Library.Main.BasicResults.AddError(String message, Exception ex)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
13 Oct 2017 18:48: Request for http://localhost:8200/api/v1/serverstate/?lasteventid=14&longpoll=true&duration=299s gave error
System.Threading.ThreadAbortException: Thread was being aborted.
   at System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, UInt32 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
   at Duplicati.Server.EventPollNotify.Wait(Int64 eventId, Int32 milliseconds)
   at Duplicati.Server.WebServer.RESTMethods.RequestInfo.LongPollCheck(EventPollNotify poller, Int64& id, Boolean& isError)
   at Duplicati.Server.WebServer.RESTMethods.ServerState.GET(String key, RequestInfo info)
   at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
13 Oct 2017 18:48: Request for http://localhost:8200/api/v1/serverstate/?lasteventid=14&longpoll=true&duration=299s gave error
System.Threading.ThreadAbortException: Thread was being aborted.
   at System.Threading.WaitHandle.WaitOneNative(SafeHandle waitableSafeHandle, UInt32 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.InternalWaitOne(SafeHandle waitableSafeHandle, Int64 millisecondsTimeout, Boolean hasThreadAffinity, Boolean exitContext)
   at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
   at Duplicati.Server.EventPollNotify.Wait(Int64 eventId, Int32 milliseconds)
   at Duplicati.Server.WebServer.RESTMethods.RequestInfo.LongPollCheck(EventPollNotify poller, Int64& id, Boolean& isError)
   at Duplicati.Server.WebServer.RESTMethods.ServerState.GET(String key, RequestInfo info)
   at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
13 Oct 2017 18:48: Reporting error gave error
System.ObjectDisposedException: Cannot write to a closed TextWriter.
   at System.IO.__Error.WriterClosed()
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)
13 Oct 2017 18:48: Reporting error gave error
System.ObjectDisposedException: Cannot write to a closed TextWriter.
   at System.IO.__Error.WriterClosed()
   at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
   at Duplicati.Server.WebServer.RESTHandler.DoProcess(RequestInfo info, String method, String module, String key)

JonMikelV · October 15, 2017, 12:48am

Normally Duplicati is pretty reliable and very robust when dealing with restoring from partial backups (such as a cloud provider lost some of you backup files).

That being said, there are some known issues, many of which are being addressed in upcoming releases. One of these is restore UI performance which seems to be related to number of records in the sqlite database.

I don’t know if it’s a large number of files, blocks, or both that does it but do you know the approximate total file count and size of the data being backed up?

Also, do you know if your C:, D:, and E: drives are all formatted the same?

kenkendk · October 16, 2017, 2:24pm

I noticed that you wrote “CrashPlan” user. Maybe you are seeing this problem:

frankieandshadow · October 17, 2017, 10:35am

That could indeed be the problem.

Unfortunately this is a bit of a dilemma. I am not in a position to turn CrashPlan off until I can get an alternative solution working.

The thread you references says “most people will probably not have a backup of AppData\Local running”. I disagree. If you backup c:\Users\USER, which is the natural thing to do, you will get that backed up.

But in any case, CrashPlan is just an example - other programs may do the same. I will follow Daniel_Eriksson’s solution for now, but I think the long term remedy is for Duplicati to recognise this can happen and retry periodically until the lock is freed.

I imaging the backup is taking a lock in order to try to ensure the integrity of the file in the backup.

frankieandshadow · October 27, 2017, 9:15am

This has now run to completion, and also iover SSH, so that might have been the same problem. To Backblaze it hasn’t but that’s a different problem which I’ll put in a new thread.