Backblaze b2 times out

I am using the 2.0.5.114_canary version and starting June 1st, the backups timeout as soon as they try to connect to B2’s servers. The error message is the following:

System.AggregateException: One or more errors occurred. (Error: ConnectFailure (Connection timed out) (Error: ConnectFailure (Connection timed out)) (One or more errors occurred. (Error: ConnectFailure (Connection timed out)))) ---> System.AggregateException: Error: ConnectFailure (Connection timed out) (Error: ConnectFailure (Connection timed out)) (One or more errors occurred. (Error: ConnectFailure (Connection timed out))) ---> System.Net.WebException: Error: ConnectFailure (Connection timed out) ---> System.Net.Sockets.SocketException: Connection timed out
at System.Net.Sockets.SocketAsyncResult.CheckIfThrowDelayedException () [0x00014] in <1af2426e91d1474d92e5d471c4ca8f95>:0
at System.Net.Sockets.Socket.EndConnect (System.IAsyncResult asyncResult) [0x0002c] in <1af2426e91d1474d92e5d471c4ca8f95>:0
at System.Net.WebConnection+<>c.<Connect>b__16_1 (System.IAsyncResult asyncResult) [0x00006] in <1af2426e91d1474d92e5d471c4ca8f95>:0
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x00019] in <efe941bb62534dc3a62ceb1a818964a0>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Net.WebConnection.Connect (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x001e1] in <1af2426e91d1474d92e5d471c4ca8f95>:0
--- End of inner exception stack trace ---

I also do other tasks with rclone and it can connect just fine.

Has anyone noticed the same issue? What should I try?

I actually have another backup destinations at b2 and only one of the two shows this behavior.
I tried a repair --dry-run on the problematic one, and it fails with the same error, after 5 lines of Listing remote folder ...

Could it come from the number of files in the bucket?

What OS, and if not Windows, what does mono --version say?

What To Do When You Get a B2 503 (or 500) Server Error says how original B2 (not the S3 API) directs uploads to a specific piece of equipment, which might explain why your results are varying. You can see the connections (and sometimes the attempts) in netstat or other tools, depending on what OS this is.

You can try to isolate the problem by using the Test connection button, which does only a list request, however you found some other evidence (discussed below) that suggests that a list operation will fail.

If you Export As Command-line, you can use that URL with Duplicati.CommandLine.BackendTool.exe to manually list or get files. If you’re going to put, it would be safer to the backup to use a different folder.

If you wind up making a new folder, you can also use Duplicati.CommandLine.BackendTester.exe to test.

Backblaze seems to suffer lots from security software blocklist interference. Test without such things on.

Is this CLI or are you reading logs? Regardless, number-of-retries defaults to 5. So I guess list fails.
It is conceivable that some other operations work. I don’t know if B2 is bothered by the number of files.

Are the two buckets located on the same destination/country?

B2 Buckets located in US are slow when accessed from EU - compared to their Dutch datacenter accessed within EU.

Well, it all boils down to the value of the --asynchronous-concurrent-upload-limit parameter.
It used to be set at 20 and working just fine. But after after a bit of experimentation, it seems anything above 10 is now rejected by Backblaze.

This is quite strange because I’m using rclone with --transfers set to 32 and it works just fine. But maybe it’s more resilient to some connections being killed/timed out.

As to your questions, I’m running on a Linux Arch based OS, on the command line and mono is at version 6.12.0. The web UI was giving the same issue, also on the Test connection button.
Finally, the buckets are all in the same region.

Interesting, but it was looking like you weren’t even getting connected (to be rejected) unless they’ve got some network-level rejection set up, which I suppose they could do to mitigate attacks or even high use

If you’re good at capturing packets with tcpdump, Wireshark, etc., you could probably get a better look…

On a simpler networking note, if you’re truly being ignored at a TCP level, netstat will show a SYN_SENT.
I’m not an expert at lsof, but if you have it then it looks like sudo lsof -iTCP -sTCP:SYN_SENT is easier.

I had the same problem
The firewall has blocked www.backblazeb2.com

1 Like

Welcome to the forum @Ramez_Mazzawi

Thanks for suggesting a possible explanation. Backblaze seems to get blocked an unusually large amount.
The exact URL varies. Actually, are you sure you got the right one? I don’t see that as being a known name.

I actually had this problem with the firewall of checkpoint And there would appear this address that it is blocked