To start with, I really like Duplicati! It’s really nice to have it in the background on my Debian server, and the backups just work… well, some of them…
The cloud service with the most space is my OneDrive for Business (about 1 TB), but it just does not work for me.
I have read all the tips about how you “sign in”, testing the connection works, Duplicati manages to write files, but the verification just does not work:
Cut out from result log:
MainOperation: Test Verifications: [ Key: duplicati-20171121T204420Z.dlist.zip.aes Value: [ Key: Error Value: The request timed out ]
Errors: [ Failed to process file duplicati-20171121T204420Z.dlist.zip.aes => The request timed out,
All my other services work well: OneDrive Personal, G-Drive, BackBlaze, Mega (extreemely slow, though).
Hi @rickard - thanks for using (and liking!) Duplicati!
It sounds to me like you’ve got a slow connection and it’s Duplicati gives up waiting for OD4B to respond (hence the “timed out” error).
You didn’t mention what version of Duplicati you’re using, but if it’s one of the newer Canary versions you should have access to the --http-operation-timeout parameter which lets you extend how long Duplicati will wait for a response from your cloud provider.
Note that I believe the default timeout is 10 minutes, so if you’re finding that the error comes up 10 minutes after the last step then that’s a good confirmation we’re on the right track.
(BTW - I edited your post by putting “~~~” before and after the error message to make it easier to read.)
The edit of my post, did that create the textbox with scrollbars? (Nice!)
I am running the lastest stable version for Debian “duplicati_126.96.36.199-1_all.deb”.
Well… It’s not a slow connection, I have 100 Mbps and it works great (and fast) with OneDrive Personal, where I store 80 GB, compared to OD4B where my test files are a total of 95 MB. On the other hand, the symptoms are like timeouts, which also the error messages say.
Under “Advanced options” I find a parameter for OD4B “web-timeout”, could that be what you are thinking about?
Yep - I like Discourse (this forum tool) quite a bit.
To be honest, I’ve never used OD4B so I’ve never used it before, but from the description in the web GUI it sounds like it will likely do (just for OD4B) what the http-operation timeout does in general.
I’d say give that (--web-timeout in OD4B options) a try and see if that helps at all.
The backup operations works really fast, and then the verification phase starts, when Duplicati should fetch a file, and that’s when everything stalls…
I changed the web-timeout to 300 secs (5 mins), which sound way to much… I have verifications with 500 MB files that are a lot less than 5 mins. Never the less, it was tried.
The logs below are from different stages in the process.
System.Net.WebException: The request timed out
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00052] in <bd46d4d4f7964dfa9beea098499ab597>:0
at System.Net.HttpWebRequest.GetResponse () [0x0000e] in <bd46d4d4f7964dfa9beea098499ab597>:0
at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute () [0x00010] in <ae42712e06c349faba29591f4f24e683>:0
at Microsoft.SharePoint.Client.File.OpenBinaryDirect (Microsoft.SharePoint.Client.ClientContext context, System.String serverRelativeUrl) [0x00078] in <8e3eb3065b524a909640c2bd1341366d>:0
at Duplicati.Library.Backend.SharePointBackend.doGet (System.String remotename, System.IO.Stream stream, System.Boolean useNewContext) [0x000cd] in <1e96623231d94ecd98631cef66e60b62>:0
at Duplicati.Library.Backend.SharePointBackend.Get (System.String remotename, System.IO.Stream stream) [0x00000] in <1e96623231d94ecd98631cef66e60b62>:0
at Duplicati.Library.Main.BackendManager.coreDoGetPiping (Duplicati.Library.Main.BackendManager+FileEntryItem item, Duplicati.Library.Interface.IEncryption useDecrypter, System.Int64& retDownloadSize, System.String& retHashcode) [0x001a2] in <118ad25945a24a3991f7b65e7a45ea1e>:0
Operation Get with file duplicati-i0ec51ce95e624d979a68a8ae2511a889.dindex.zip.aes attempt 5 of 5 failed with message: The request timed out
System.Net.WebException: The request timed out
at Duplicati.Library.Main.BackendManager.GetForTesting (System.String remotename, System.Int64 size, System.String hash) [0x00054] in <118ad25945a24a3991f7b65e7a45ea1e>:0
at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x003f7] in <118ad25945a24a3991f7b65e7a45ea1e>:0
I set the “no backend verification”, and if nothing was changed it was lightning fast. When I added source files, it was slower than the other backups, but a lot faster than “normal” for OD4B. I did, however, get two warnings:
Delete operation failed for duplicati-i6224f3daf48642e596372eb096c7accf.dindex.zip.aes with FileNotFound, listing contents
Duplicati.Library.Interface.FileMissingException: Element with path '/personal/rickard************/Documents/Rickard/Backup/duplicati-i6224f3daf48642e596372eb096c7accf.dindex.zip.aes' not found on host 'https://*****-my.sharepoint.com/personal/rickard***************'.
at Duplicati.Library.Backend.SharePointBackend.wrappedExecuteQueryOnConext (Microsoft.SharePoint.Client.ClientContext ctx, System.String serverRelPathInfo, System.Boolean isFolder) [0x00064] in <1e96623231d94ecd98631cef66e60b62>:0
at Duplicati.Library.Backend.SharePointBackend.doDelete (System.String remotename, System.Boolean useNewContext) [0x00034] in <1e96623231d94ecd98631cef66e60b62>:0
Do others get OD4B to work just like the other services, or is it really tricky to get it to work?
2017-12-15 21:14:36Z - Warning: Operation List with file attempt 1 of 5 failed with message: No SharePoint web could be logged in to at path 'https://XXXXX-my.sharepoint.com/personal/XXX_XXXXX_XX/Documents/Duplicati/Test'. Maybe wrong credentials. Or try using '//' in path to separate web from folder path.
System.Net.WebException: No SharePoint web could be logged in to at path 'https://XXXXX-my.sharepoint.com/personal/XXX_XXXXX_XX/Documents/Duplicati/Test'. Maybe wrong credentials. Or try using '//' in path to separate web from folder path.
at Duplicati.Library.Backend.SharePointBackend.getSpClientContext (System.Boolean forceNewContext) [0x00070] in <7071b8cb3c53472e8633e3e37f5e18d5>:0
at Duplicati.Library.Backend.SharePointBackend+<doList>d__28.MoveNext () [0x00032] in <7071b8cb3c53472e8633e3e37f5e18d5>:0
at System.Collections.Generic.List`1[T]..ctor (System.Collections.Generic.IEnumerable`1[T] collection) [0x0008b] in <8f2c484307284b51944a1a13a14c0266>:0
at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x00011] in <63992662b765477a898ef49cdcc99ee2>:0
at Duplicati.Library.Main.BackendManager.DoList (Duplicati.Library.Main.BackendManager+FileEntryItem item) [0x0001b] in <df9891e588c04bd199c53e6d7539922f>:0
at Duplicati.Library.Main.BackendManager.ThreadRun () [0x000ee] in <df9891e588c04bd199c53e6d7539922f>:0
When browsing to the mentioned URL, I can see the files that have been uploaded.
@rickard: If it fails on Linux, but works on Windows, my guess is that is some issue with Mono. Make sure you are using the Xamarin provided version (5.x) and not the old Debian provided version (3.x).
If possible, you can also try the latest build (188.8.131.52_experimental, should be available right now), and see if some of the general updates fixes the problem.
I have a bit of trouble debugging the issue, so I hope you can try various things to find out when it breaks. I have a test account for OD4B, but I can’t start throwing multiple GB at it to see if it breaks.
Good call with the Mono library, I was running with the built in Debian default (184.108.40.206) and changed to Xamarin (220.127.116.11). However, this did not fix the issue…
I also installed the experimental Duplicati (18.104.22.168), and that did not change anything either. The behaviour is still the same, with timeouts during verification, and sometimes complete authentication errors.