For several weeks I’ve successfully used Duplicati (currently running 184.108.40.206_beta_2017-08-01) on Linux to backup a large (currently 1.13 TB) dataset to Backblaze B2. I am using a 50 MB block size. The initial backup and a few incrementals completed successfully.
After a week or so, I added another large directory to my backup set, bringing the total size to around 1.6 TB. My next backup (unsurprisingly) took several days, but it eventually failed with a ThreadAbortException. That’s not the problem, though. The problem is that I can no longer backup at all.
Each attempted backup now runs for almost exactly 10 minutes with the message “Verifying backend data”, and then fails with the error message below. Can Duplicati simply not handle this size data set? Would I be better off breaking my backup into a handful of smaller (say 500 GB) backups? Would increasing my block size help? Any other suggestions?
System.Net.WebException: GetResponse timed out ---> System.Net.WebException: Aborted. at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) <0x41599aa0 + 0x001a3> in <filename unknown>:0 at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.OnAsync (IAsyncResult r) <0x4158ee70 + 0x000eb> in <filename unknown>:0 --- End of inner exception stack trace --- at Duplicati.Library.Main.BackendManager.List () <0x415de000 + 0x0016b> in <filename unknown>:0 at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, IBackendWriter log, System.String protectedfile) <0x415dacd0 + 0x0015f> in <filename unknown>:0 at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, IBackendWriter log, System.String protectedfile) <0x415d87e0 + 0x000cb> in <filename unknown>:0 at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) <0x415d76a0 + 0x001af> in <filename unknown>:0