Out of memory since latest canary

[Wim_Jansen]

Since upgrading from beta to latest canary, I get at seemingly random moments Out of Memory errors. Anyone else experiencing this? Extract of log:

System.OutOfMemoryException: Out of memory
  at (wrapper managed-to-native) System.String:FastAllocate
tring (int)
  at System.Text.StringBuilder.ToString () [0x00017] in <8f
c484307284b51944a1a13a14c0266>:0
  at Duplicati.Library.Modules.Builtin.ResultSerialization.
uplicatiFormatSerializer.Serialize (System.Object result, S
stem.Collections.Generic.IEnumerable`1[T] loglines, System.
ollections.Generic.Dictionary`2[TKey,TValue] additional) [0
002b9] in <421679d49a6d4205a7c3ddeec540cee0>:0
  at Duplicati.Library.Modules.Builtin.ReportHelper.Replace
emplate (System.String input, System.Object result, System.
oolean subjectline) [0x00254] in <421679d49a6d4205a7c3ddeec
40cee0>:0
  at Duplicati.Library.Modules.Builtin.ReportHelper.OnFinis
 (System.Object result) [0x00124] in <421679d49a6d4205a7c3d
eec540cee0>:0
2018-11-06 22:19:57 +01 - [Error-Duplicati.Library.Main.Con
roller-FailedOperation]: The operation Backup has failed wi
h error: Object reference not set to an instance of an obje
t
System.NullReferenceException: Object reference not set to
n instance of an object
  at SharpCompress.Compressors.Deflate.ZlibBaseStream.finis
 () [0x00033] in <e590b95deffd47918abc01584b2527c9>:0
  at SharpCompress.Compressors.Deflate.ZlibBaseStream.Dispo
e (System.Boolean disposing) [0x00024] in <e590b95deffd4791
abc01584b2527c9>:0
  at System.IO.Stream.Close () [0x00000] in <8f2c484307284b
1944a1a13a14c0266>:0
  at System.IO.Stream.Dispose () [0x00000] in <8f2c48430728
b51944a1a13a14c0266>:0
  at (wrapper remoting-invoke-with-check) System.IO.Stream:
ispose ()
  at SharpCompress.Compressors.Deflate.DeflateStream.Dispos
 (System.Boolean disposing) [0x00013] in <e590b95deffd47918
bc01584b2527c9>:0
  at System.IO.Stream.Close () [0x00000] in <8f2c484307284b
1944a1a13a14c0266>:0
  at System.IO.Stream.Dispose () [0x00000] in <8f2c48430728
b51944a1a13a14c0266>:0
  at (wrapper remoting-invoke-with-check) System.IO.Stream:
ispose ()
  at SharpCompress.Writers.Zip.ZipWriter+ZipWritingStream.D
spose (System.Boolean disposing) [0x0001d] in <e590b95deffd
7918abc01584b2527c9>:0
  at System.IO.Stream.Close () [0x00000] in <8f2c484307284b
1944a1a13a14c0266>:0
  at System.IO.StreamWriter.Dispose (System.Boolean disposi
g) [0x0004d] in <8f2c484307284b51944a1a13a14c0266>:0
  at System.IO.StreamWriter.Close () [0x00000] in <8f2c4843
7284b51944a1a13a14c0266>:0
  at Newtonsoft.Json.JsonTextWriter.CloseBufferAndWriter ()
[0x00032] in <dc86da7fc46c487ba6c7ab826da479cc>:0
  at Newtonsoft.Json.JsonTextWriter.Close () [0x00006] in <
c86da7fc46c487ba6c7ab826da479cc>:0
  at Duplicati.Library.Main.Volumes.FilesetVolumeWriter.Clo
e () [0x00013] in <f160005497df4b93a76541b8b0fb83ee>:0
  at Duplicati.Library.Main.Volumes.FilesetVolumeWriter.Dis
ose () [0x00000] in <f160005497df4b93a76541b8b0fb83ee>:0
  at Duplicati.Library.Main.Operation.BackupHandler+<RunAsy
c>d__19.MoveNext () [0x00fb0] in <f160005497df4b93a76541b8b
fb83ee>:0
--- End of stack trace from previous location where excepti
n was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo
Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0
  at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Thr
ading.Tasks.Task task) [0x00050] in <6973ce2780de4b28aaa2c5
fc59993b1>:0
  at Duplicati.Library.Main.Operation.BackupHandler.Run (Sy
tem.String[] sources, Duplicati.Library.Utility.IFilter fil
er) [0x00008] in <f160005497df4b93a76541b8b0fb83ee>:0
  at Duplicati.Library.Main.Controller+<>c__DisplayClass13_
.<Backup>b__0 (Duplicati.Library.Main.BackupResults result)
[0x00035] in <f160005497df4b93a76541b8b0fb83ee>:0
  at Duplicati.Library.Main.Controller.RunAction[T] (T resu
t, System.String[]& paths, Duplicati.Library.Utility.IFilte
& filter, System.Action`1[T] method) [0x0011d] in <f1600054
7df4b93a76541b8b0fb83ee>:0

[JonMikelV]

Not I (yet).

Am I reading that error correctly in that it seems to be related to compression (DeflateStream.Dispose) as part of the FilesetVolumeWriter.Close method?

[Pectojin]

It could be related to multi threading. More concurrent compressions will require more memory.

The concurrent compression can be set with --concurrency-compressors, which defaults to 2.

[JonMikelV]

Good point - I think I’ve still got all my multi-threading parameters set to disable it (left over from older canary issues).

[Wim_Jansen]

@Pectojin

The concurrent compression can be set with --concurrency-compressors , which defaults to 2.

trying this now, was thinking about the concurrency after the post

[kenkendk]

@Wim_Jansen

Extract of log:

Reading the exception, it seems that the problem is that your results are simply too large. I did not anticipate this, so the current way it does this is a simple text replace, so you need to have the entire results object as text in memory. If you have a lot of log lines to include, this could potentially be a large chunk of contiguous memory.

Not sure why it is attempting to compress this message though. But it could be because something else fails that it gets a null error.

[Wim_Jansen]

I have had other errors last days which might be related or not. I moved one backup from rclone to od4b so I was thinking the error came from the backend. Moved back to rclone and now disabled multithreading. Will wait for next schedule this night. If all goes well I’ll first move back to od4b. If that goes well, will enable multithreading again and see if error comes back.

The memory error was not appearing always. I would need time to test but it could be it only appears if more than 1 dblock (500mb) is needed on my ds216j with only 512mb RAM.

Other error:

Failed: Value cannot be null.
Parameter name: dest
Details: System.ArgumentNullException: Value cannot be null.
Parameter name: dest
at System.IO.MemoryStream.set_Capacity (System.Int32 value) [0x0001c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.MemoryStream.EnsureCapacity (System.Int32 value) [0x00076] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.MemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x000cb] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpContent+FixedMemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00007] in <41c316444c8a4380b9344ba09eeb18de>:0 
at System.IO.MemoryStream.WriteAsync (System.Byte[] buffer, System.Int32 offset, System.Int32 count, System.Threading.CancellationToken cancellationToken) [0x00082] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.Stream+<CopyToAsyncInternal>c__async0.MoveNext () [0x0009b] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpContent+<LoadIntoBufferAsync>c__async1.MoveNext () [0x000cb] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00196] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.CompactHandler.DoCompact (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Boolean hasVerifiedBackend, System.Data.IDbTransaction& transaction, Duplicati.Library.Main.BackendManager sharedBackend) [0x00268] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBacked, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x00399] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired (Duplicati.Library.Main.BackendManager backend, System.Int64 lastVolumeSize) [0x000a5] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x00db7] in <f160005497df4b93a76541b8b0fb83ee>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <6973ce2780de4b28aaa2c5ffc59993b1>:0 
at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00008] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Controller+<>c__DisplayClass13_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x00035] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0011d] in <f160005497df4b93a76541b8b0fb83ee>:0 

Log data:
2018-11-05 20:26:05 +01 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
System.ArgumentNullException: Value cannot be null.
Parameter name: dest
at System.IO.MemoryStream.set_Capacity (System.Int32 value) [0x0001c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.MemoryStream.EnsureCapacity (System.Int32 value) [0x00076] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.MemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x000cb] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpContent+FixedMemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00007] in <41c316444c8a4380b9344ba09eeb18de>:0 
at System.IO.MemoryStream.WriteAsync (System.Byte[] buffer, System.Int32 offset, System.Int32 count, System.Threading.CancellationToken cancellationToken) [0x00082] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.IO.Stream+<CopyToAsyncInternal>c__async0.MoveNext () [0x0009b] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpContent+<LoadIntoBufferAsync>c__async1.MoveNext () [0x000cb] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
at System.Net.Http.HttpClient+<SendAsyncWorker>c__async0.MoveNext () [0x00196] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.CompactHandler.DoCompact (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Boolean hasVerifiedBackend, System.Data.IDbTransaction& transaction, Duplicati.Library.Main.BackendManager sharedBackend) [0x00268] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBacked, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x00399] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired (Duplicati.Library.Main.BackendManager backend, System.Int64 lastVolumeSize) [0x000a5] in <f160005497df4b93a76541b8b0fb83ee>:0 
at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x00db7] in <f160005497df4b93a76541b8b0fb83ee>:0

[Wim_Jansen]

last schedule ran fine, but when removing the concurrency-parameter (setting to the default value), gave below error:

{"ClassName":"System.ArgumentNullException",
"Message":"Value cannot be null.","Data":null,
"InnerException":null,
"HelpURL":null,
"StackTraceString":" at System.IO.MemoryStream.set_Capacity (System.Int32 value) [0x0001c] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.IO.MemoryStream.EnsureCapacity (System.Int32 value) [0x00076] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.IO.MemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x000cb] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Net.Http.HttpContent+FixedMemoryStream.Write (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00007] in <41c316444c8a4380b9344ba09eeb18de>:0 
 at System.IO.MemoryStream.WriteAsync (System.Byte[] buffer, System.Int32 offset, System.Int32 count, System.Threading.CancellationToken cancellationToken) [0x00082] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.IO.Stream+&lt;CopyToAsyncInternal>c__async0.MoveNext () [0x0009b] in <8f2c484307284b51944a1a13a14c0266>:0 
--- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Net.Http.HttpContent+&lt;LoadIntoBufferAsync>c__async1.MoveNext () [0x000cb] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0004e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x0002e] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x0000b] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Runtime.CompilerServices.ConfiguredTaskAwaitable+ConfiguredTaskAwaiter.GetResult () [0x00000] in <8f2c484307284b51944a1a13a14c0266>:0 
 at System.Net.Http.HttpClient+&lt;SendAsyncWorker>c__async0.MoveNext () [0x00196] in <41c316444c8a4380b9344ba09eeb18de>:0 
--- End of stack trace from previous location where exception was thrown ---
 at Duplicati.Library.Main.BackendManager.GetForTesting (System.String remotename, System.Int64 size, System.String hash) [0x00065] in <f160005497df4b93a76541b8b0fb83ee>:0 
 at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x0042f] in <f160005497df4b93a76541b8b0fb83ee>:0 ",
"RemoteStackTraceString":null,
"RemoteStackIndex":0,"ExceptionMethod":null,
"HResult":-2147467261,"Source":"Duplicati.Library.Main",
"ParamName":"dest"}

[JonMikelV]

Thanks for helping narrow down the memory issue to concurrency as @Pectojin posited!

BTW - I edited your post and added “~~~ json” and “~~~” before & after your message to make the error message more readable.

[Pectojin]

@Wim_Jansen

The memory error was not appearing always. I would need time to test but it could be it only appears if more than 1 dblock (500mb) is needed on my ds216j with only 512mb RAM.

Is duplicati running on the ds216j or is it just the backend?

I guess what I’m asking is if the concurrency settings should adjust based on system memory. We could calculate the required memory vs the system memory and scale down if threads aren’t explicitly configured by the user.

[Wim_Jansen]

It is running on the ds216j.

[Pectojin]

Ok, so that change would actually fix this issue

It’s always good to figure out if it’s gonna help someone before writing the code

[Pectojin]

I created an issue on Github to track that enhancement: [Enhancement] Determine --concurrency-compressors based on memory requirements · Issue #3478 · duplicati/duplicati · GitHub

[Wim_Jansen]

Server bricked again… not sure about the error, no time to have a look now. It no longer responds on 8200.

[Wim_Jansen]

Error message (Not sure I have everything copied, mobile layout )

System.ArgumentNullException: Array cannot be null. Parameter name: bytes
 at System.Text.UTF8Encoding.GetBytes (System.String s, System.Int32 charIndex, System.Int32 charCount, System.Byte[] bytes, System.Int32 byteIndex) [0x0002c] in <8f2c484307284b51944a1a13a14c0266>:0
 at Mono.Data.Sqlite.SqliteConvert.ToUTF8 (System.String sourceText) [0x00022] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SQLite3.Bind_Text (Mono.Data.Sqlite.SqliteStatement stmt, System.Int32 index, System.String value) [0x00000] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SqliteStatement.BindParameter (System.Int32 index, Mono.Data.Sqlite.SqliteParameter param) [0x001c1] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SqliteStatement.BindParameters () [0x0001c] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SqliteCommand.GetStatement (System.Int32 index) [0x00049] in <62e44d67e13348a795e30fd7dd8622df>:0
 at (wrapper remoting-invoke-with-check) Mono.Data.Sqlite.SqliteCommand:GetStatement (int)
 at Mono.Data.Sqlite.SqliteDataReader.NextResult () [0x000cc] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SqliteDataReader..ctor (Mono.Data.Sqlite.SqliteCommand cmd, System.Data.CommandBehavior behave) [0x00051] in <62e44d67e13348a795e30fd7dd8622df>:0
 at (wrapper remoting-invoke-with-check) Mono.Data.Sqlite.SqliteDataReader:.ctor (Mono.Data.Sqlite.SqliteCommand,System.Data.CommandBehavior)
 at Mono.Data.Sqlite.SqliteCommand.ExecuteReader (System.Data.CommandBehavior behavior) [0x00006] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Mono.Data.Sqlite.SqliteCommand.ExecuteNonQuery () [0x00000] in <62e44d67e13348a795e30fd7dd8622df>:0
 at Duplicati.Server.Database.Connection.OverwriteAndUpdateDb[T] (System.Data.IDbTransaction transaction, System.String deleteSql, System.Object[] deleteArgs, System.Collections.Generic.IEnumerable`1[T] values, System.String insertSql, System.Func`2[T,TResult] f) [0x000e8] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Database.Connection.SetSettings (System.Collections.Generic.IEnumerable`1[T] values, System.Int64 id, System.Data.IDbTransaction transaction) [0x00030] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Database.ServerSettings.SaveSettings () [0x00000] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Database.ServerSettings.set_UnackedError (System.Boolean value) [0x00039] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Database.Connection.RegisterNotification (Duplicati.Server.Serialization.NotificationType type, System.String title, System.String message, System.Exception ex, System.String backupid, System.String action, System.String logid, System.String messageid, System.String logtag, System.Func`3[T1,T2,TResult] conflicthandler) [0x000e2] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Runner.UpdateMetadataError (Duplicati.Server.Serialization.Interface.IBackup backup, System.Exception ex) [0x000dc] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00614] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Server.Program+<>c.<RealMain>b__41_2 (Duplicati.Server.Runner+IRunnerData x) [0x00000] in <92818737726840c28e3c9d07cee8c85b>:0
 at Duplicati.Library.Utility.WorkerThread`1[Tx].Runner () [0x00191] in <9feca61547df434cab9f9628c43716f5>:0

[JonMikelV]

Definitely seems a different error from last time, but what it’s complaining about I’m not sure.

(BTW - to ease my digging through the error I edited your post by adding line breaks and wrapping it in “~~~”.)