Error uploading to B2 - One or more errors occurred. (The socket has been shut down )

Support
1

Hi,
This is my first day using Duplicati. I’m trying to backup to B2 based storage. When I run the job, it seems to start off fine, then it stops with the following error:

Error while running Backup
One or more errors occurred. (The socket has been shut down (The socket has been shut down) (One or more errors occurred. (The socket has been shut down)))

I’ve tried creating a new bucket, and adjusting various settings, but I can’t seem to get it to work. Any suggestions?

2

Are you using the free tier? If so maybe you are hitting the 10GB limit?

Or do you use Pi-hole? I have seen some people have trouble with B2 because Pi-hole was denying DNS resolution for some B2 hosts.

1 Like
3

Hi Drwtsn32,
Thanks for your thoughts. I am using the free tier (for the moment), but
a) no files have been uploaded so far. It seems to fail before even a single successful upload; and
b) the test I’m doing is with only 3GB, so well below the limit.

I’m not using Pi-hole, but I do use Ubiquity Unifi. I’ve checked the Unifi logs and don’t see anything odd happening there.

4

PS, here’s some more log data in case it’s helpful

System.AggregateException: One or more errors occurred. (The socket has been shut down (The socket has been shut down) (One or more errors occurred. (The socket has been shut down))) ---> System.AggregateException: The socket has been shut down (The socket has been shut down) (One or more errors occurred. (The socket has been shut down)) ---> System.Net.Sockets.SocketException: The socket has been shut down
  at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType12`1[<Input>j__TPar] self) [0x005ac] in <e60bc008dd1b454d861cfacbdd3760b9>:0 
  at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType12`1[<Input>j__TPar] self) [0x007d7] in <e60bc008dd1b454d861cfacbdd3760b9>:0 
  at CoCoL.AutomationExtensions.RunTask[T] (T channels, System.Func`2[T,TResult] method, System.Boolean catchRetiredExceptions) [0x000d5] in <9a758ff4db6c48d6b3d4d0e5c2adf6d1>:0 
  at Duplicati.Library.Main.Operation.BackupHandler.FlushBackend (Duplicati.Library.Main.BackupResults result, CoCoL.IWriteChannel`1[T] uploadtarget, System.Threading.Tasks.Task uploader) [0x000f2] in <e60bc008dd1b454d861cfacbdd3760b9>:0 ```
5

Is that the complete log? I’m not really seeing why the socket has been shut down.

Maybe the Live Log will have more info. Open a tab to Duplicati web UI, click About → Show Log → Live → and set the dropdown to Warning. Leave this tab here and open another tab to Duplicati. Kick off the backup job and then switch back to the first tab. Let us know what warnings/errors you see in the Live Log.

1 Like
6

Thanks for the great suggestion. I didn’t know about the live log feature. I’ve set the logging up to Info level and watched as it tried to do a backup. It seems like pushing the files to the backend is having some problems. It tried over and over with different names before giving up.

Here’s the full log of the event when it gave up:

3 Sep 2021 09:37: Operation Put with file duplicati-b0ef3ed00ddf64b3e88359f3102882ccf.dblock.zip.aes attempt 6 of 6 failed with message: Unable to read data from the transport connection: The socket has been shut down.
{"ClassName":"System.IO.IOException","Message":"Unable to read data from the transport connection: The socket has been shut down.","Data":null,"InnerException":{"NativeErrorCode":10058,"ClassName":"System.Net.Sockets.SocketException","Message":"The socket has been shut down","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"  at Duplicati.Library.Main.Operation.Backup.BackendUploader.<Run>b__13_0 (<>f__AnonymousType12`1[<Input>j__TPar] self) [0x005ac] in <e60bc008dd1b454d861cfacbdd3760b9>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467259,"Source":"mscorlib"},"HelpURL":null,"StackTraceString":"  at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.ThrowException (System.Net.Sockets.SocketError error) [0x00007] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult (System.Int16 token) [0x0001f] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at System.Threading.Tasks.ValueTask+ValueTaskSourceAsTask+<>c.<.cctor>b__4_0 (System.Object state) [0x00030] in <36514b14425c4337b446653c547aa9c3>:0 \n--- End of stack trace from previous location where exception was thrown ---\n\n  at Mono.Net.Security.MobileAuthenticatedStream.InnerWrite (System.Boolean sync, System.Threading.CancellationToken cancellationToken) [0x000d3] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x00199] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x0008b] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at Mono.Net.Security.MobileAuthenticatedStream.StartOperation (Mono.Net.Security.MobileAuthenticatedStream+OperationType type, Mono.Net.Security.AsyncProtocolRequest asyncRequest, System.Threading.CancellationToken cancellationToken) [0x0024b] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at System.Net.WebRequestStream.ProcessWrite (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Threading.CancellationToken cancellationToken) [0x00270] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at System.Net.WebRequestStream.WriteAsyncInner (System.Byte[] buffer, System.Int32 offset, System.Int32 size, System.Net.WebCompletionSource completion, System.Threading.CancellationToken cancellationToken) [0x001aa] in <4f55bbf82f9b4c60960312888c59bd5c>:0 \n  at Duplicati.Library.Utility.Utility.CopyStreamAsync (System.IO.Stream source, System.IO.Stream target, System.Boolean tryRewindSource, System.Threading.CancellationToken cancelToken, System.Byte[] buf) [0x00166] in <2a3ee711c7c04f6c957360f2cf183a7f>:0 \n  at Duplicati.Library.Utility.Utility.CopyStreamAsync (System.IO.Stream source, System.IO.Stream target, System.Threading.CancellationToken cancelToken) [0x00077] in <2a3ee711c7c04f6c957360f2cf183a7f>:0 \n  at Duplicati.Library.Backend.Backblaze.B2+<>c__DisplayClass28_0.<PutAsync>b__1 (Duplicati.Library.Utility.AsyncHttpRequest req, System.Threading.CancellationToken reqCancelToken) [0x0008a] in <f30a9ba7585445e094ae4320fb244dfc>:0 \n  at Duplicati.Library.JSONWebHelper.GetJSONDataAsync[T] (System.String url, System.Threading.CancellationToken cancelToken, System.Action`1[T] setup, System.Func`3[T1,T2,TResult] setupbodyreq) [0x000c1] in <fd6b26d6c1004933aab594493968119c>:0 \n  at Duplicati.Library.Backend.Backblaze.B2.PutAsync (System.String remotename, System.IO.Stream stream, System.Threading.CancellationToken cancelToken) [0x003c7] in <f30a9ba7585445e094ae4320fb244dfc>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader.DoPut (Duplicati.Library.Main.Operation.Common.BackendHandler+FileEntryItem item, Duplicati.Library.Interface.IBackend backend, System.Threading.CancellationToken cancelToken) [0x00426] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader+<>c__DisplayClass20_0.<UploadFileAsync>b__0 () [0x0010a] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader.DoWithRetry (System.Func`1[TResult] method, Duplicati.Library.Main.Operation.Common.BackendHandler+FileEntryItem item, Duplicati.Library.Main.Operation.Backup.BackendUploader+Worker worker, System.Threading.CancellationToken cancelToken) [0x0017c] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader.DoWithRetry (System.Func`1[TResult] method, Duplicati.Library.Main.Operation.Common.BackendHandler+FileEntryItem item, Duplicati.Library.Main.Operation.Backup.BackendUploader+Worker worker, System.Threading.CancellationToken cancelToken) [0x003a5] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader.UploadFileAsync (Duplicati.Library.Main.Operation.Common.BackendHandler+FileEntryItem item, Duplicati.Library.Main.Operation.Backup.BackendUploader+Worker worker, System.Threading.CancellationToken cancelToken) [0x000da] in <e60bc008dd1b454d861cfacbdd3760b9>:0 \n  at Duplicati.Library.Main.Operation.Backup.BackendUploader.UploadBlockAndIndexAsync (Duplicati.Library.Main.Operation.Backup.VolumeUploadRequest upload, Duplicati.Library.Main.Operation.Backup.BackendUploader+Worker worker, System.Threading.CancellationToken cancelToken) [0x00098] in <e60bc008dd1b454d861cfacbdd3760b9>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146232800,"Source":"mscorlib"}
7

Welcome to the forum @mgrosvenor

Does that imply that there’s a blocker there (know what?) and any blocking is known to show in the log?

The stack trace says this is mono. What does mono --version show, and what OS is this running on?

Does the “Test connection” button on destination screen succeed? For better testing you can try running
Duplicati.CommandLine.BackendTester.exe with a URL from Export As Command-line and edited to use some empty folder. If you like, you can do manual testing with Duplicati.CommandLine.BackendTool.exe.

Are you somewhat familiar with TCP/IP networking? This will possibly wind up as a network investigation.

1 Like
8

Thanks for your suggestions.

Does that imply that there’s a blocker there (know what?)

Ubiquity Unifi is a prosumer/SME networking product suite, focused on wireless access points and security. Sort of like Cisco Meraki if you’ve heard of that. It does come with a bunch of security and intrusion protection features, but, I’ve disabled all of them to no effect.

The stack trace says this is mono. What does mono --version show, and what OS is this running on?

Running the latest and greatest Mono (6.12) on Mac OS Big Sur (11.2.3)

Does the “Test connection” button on destination screen succeed?
Yes it does.

For better testing you can try running …

Thanks for the great suggestion. I’ll take a look, but … I actually have another test I want to do first before blaming Duplicati. This may be a configuration / Backlabze issue. I tried another backup package and it also failed in a peculiar way.

Are you somewhat familiar with TCP/IP networking?

Unfortunately yes. In my other life I write high performance networking software… :-/

9

Ah ha! I’ve figured it out.

I ran a test with the B2 CLI client.

./b2-darwin upload-file test-bucket test.zip test.zip

This failed with the following error

ERROR: unauthorized for application key with capabilities ‘listBuckets,readBuckets,listFiles,readFiles,shareFiles,writeFiles,deleteFiles,readBucketEncryption,writeBucketEncryption’, restricted to bucket ‘test-bucket’, restricted to files that start with ‘test_’ (unauthorized)

It turns out, when I configured the application access key on B2, there was an option

File name prefix:
(optional)

I had assumed that this option would prepend each uploaded file with the prefix. In actual fact, it restricts files to only those with the prefix. So, by removing this option, I can now get Duplicati to work correctly.

However … the error message “socket has been shut down” was completely unhelpful in diagnosing the error. The message returned by B2 was pretty clear about what the problem is “restricted to files that start with ‘test_’ (unauthorized)”. So it seems like there might be room to improve the error reporting from Duplicati to better handle these sorts of cases.

1 Like
10

Glad you figured it out! I am a B2 user but have never experimented with that option. Thanks for following up with the solution. Never know who it might help in the future.

1 Like