Errors backing up to OneDrive for Business

Hello @JonMikelV!

The edit of my post, did that create the textbox with scrollbars? (Nice!)

I am running the lastest stable version for Debian “duplicati_2.0.2.1-1_all.deb”.

Well… It’s not a slow connection, I have 100 Mbps and it works great (and fast) with OneDrive Personal, where I store 80 GB, compared to OD4B where my test files are a total of 95 MB. On the other hand, the symptoms are like timeouts, which also the error messages say.

Under “Advanced options” I find a parameter for OD4B “web-timeout”, could that be what you are thinking about?

Rickard

Yep - I like Discourse (this forum tool) quite a bit. :slight_smile:

To be honest, I’ve never used OD4B so I’ve never used it before, but from the description in the web GUI it sounds like it will likely do (just for OD4B) what the http-operation timeout does in general.

I’d say give that (--web-timeout in OD4B options) a try and see if that helps at all.

Hi again!

The backup operations works really fast, and then the verification phase starts, when Duplicati should fetch a file, and that’s when everything stalls…

I changed the web-timeout to 300 secs (5 mins), which sound way to much… I have verifications with 500 MB files that are a lot less than 5 mins. Never the less, it was tried.

The logs below are from different stages in the process.

System.Net.WebException: The request timed out
  at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00052] in <bd46d4d4f7964dfa9beea098499ab597>:0 
  at System.Net.HttpWebRequest.GetResponse () [0x0000e] in <bd46d4d4f7964dfa9beea098499ab597>:0 
  at Microsoft.SharePoint.Client.SPWebRequestExecutor.Execute () [0x00010] in <ae42712e06c349faba29591f4f24e683>:0 
  at Microsoft.SharePoint.Client.File.OpenBinaryDirect (Microsoft.SharePoint.Client.ClientContext context, System.String serverRelativeUrl) [0x00078] in <8e3eb3065b524a909640c2bd1341366d>:0 
  at Duplicati.Library.Backend.SharePointBackend.doGet (System.String remotename, System.IO.Stream stream, System.Boolean useNewContext) [0x000cd] in <1e96623231d94ecd98631cef66e60b62>:0 
  at Duplicati.Library.Backend.SharePointBackend.Get (System.String remotename, System.IO.Stream stream) [0x00000] in <1e96623231d94ecd98631cef66e60b62>:0 
  at Duplicati.Library.Main.BackendManager.coreDoGetPiping (Duplicati.Library.Main.BackendManager+FileEntryItem item, Duplicati.Library.Interface.IEncryption useDecrypter, System.Int64& retDownloadSize, System.String& retHashcode) [0x001a2] in <118ad25945a24a3991f7b65e7a45ea1e>:0 


Operation Get with file duplicati-i0ec51ce95e624d979a68a8ae2511a889.dindex.zip.aes attempt 5 of 5 failed with message: The request timed out
System.Net.WebException: The request timed out
  at Duplicati.Library.Main.BackendManager.GetForTesting (System.String remotename, System.Int64 size, System.String hash) [0x00054] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x003f7] in <118ad25945a24a3991f7b65e7a45ea1e>:0 

Rickard

Thanks for doing that test.

Just to confirm the error is where we think it is, does the job run without errors if you set --backup-test-samples=0 or --no-backend-verification=true (checked)?

Not that you necessarily want to run that way, but it does confirm where the problem lies.

Hi!

I set the “no backend verification”, and if nothing was changed it was lightning fast. When I added source files, it was slower than the other backups, but a lot faster than “normal” for OD4B. I did, however, get two warnings:

Delete operation failed for duplicati-i6224f3daf48642e596372eb096c7accf.dindex.zip.aes with FileNotFound, listing contents
Duplicati.Library.Interface.FileMissingException: Element with path '/personal/rickard************/Documents/Rickard/Backup/duplicati-i6224f3daf48642e596372eb096c7accf.dindex.zip.aes' not found on host 'https://*****-my.sharepoint.com/personal/rickard***************'.
  at Duplicati.Library.Backend.SharePointBackend.wrappedExecuteQueryOnConext (Microsoft.SharePoint.Client.ClientContext ctx, System.String serverRelPathInfo, System.Boolean isFolder) [0x00064] in <1e96623231d94ecd98631cef66e60b62>:0 
  at Duplicati.Library.Backend.SharePointBackend.doDelete (System.String remotename, System.Boolean useNewContext) [0x00034] in <1e96623231d94ecd98631cef66e60b62>:0 

Do others get OD4B to work just like the other services, or is it really tricky to get it to work?

Well, as of yesterday 1,046 backups were saved to OneDrive for Business so I’m guessing there’s something specific to your setup / connection that is causing the issue. :frowning:

Regarding the new FileMissingException error, I may have to defer to somebody more familiar with that side of things…perhaps @kenkendk has some thoughts.

Yes… that’s right, you have the usage statistics! :slight_smile: Then we know that it works in general, which is good!

I can also run a test from my workstation tonight, and see if the Windows version behaves diffently.

Hello!

On my Win10 workstation, I have tested “duplicati-2.0.2.1_beta_2017-08-01-x64”, and here the OD4B seems to work! Seemingly a little slower than the other cloud services, but no warnings and errors!

So, I take it we have verified that my network is OK, and that there is a difference in the two duplicati versions, right?

I think my issue is the same as this bug on Github, which seems to not be solved: “Restoring from OneDrive for Business #2475”?

@rickard, unfortunately that issue (I added the link to your post, BTW - hope you don’t mind) is beyond my expertise so I’m going to put a shout out to @kenkendk for some input.

It makes my post look more professional, so thanks… :slight_smile:

i was also trying to switch to onedrive for business, but can’t seem to be able to connect (on synology). However, I need to try a bit more.

I see OneDrive has OAUTH authentication in Duplicati. Would that be possible for OD4B too? We are using a SSO login, which probably doesn’t work in the normal login procedures.

I did get OD4B to work with rclone using OAUTH, so it should be possible to do that for Duplicati as well.

There is some file listing issues in od4b above 5000

2 odb4 backups stopped working with errors like “Found 5011 file missing… Please run repair”

But with rclone all files were downloaded successful.

I’ll do some more experiments later

Yes, same here… Backup seems to work OK, verifying takes a long time and does not download anything. It’s now 7 minutes at verifying. Nothing happens.

2017-12-15 21:14:36Z - Warning: Operation List with file  attempt 1 of 5 failed with message: No SharePoint web could be logged in to at path 'https://XXXXX-my.sharepoint.com/personal/XXX_XXXXX_XX/Documents/Duplicati/Test'. Maybe wrong credentials. Or try using '//' in path to separate web from folder path.
System.Net.WebException: No SharePoint web could be logged in to at path 'https://XXXXX-my.sharepoint.com/personal/XXX_XXXXX_XX/Documents/Duplicati/Test'. Maybe wrong credentials. Or try using '//' in path to separate web from folder path.
  at Duplicati.Library.Backend.SharePointBackend.getSpClientContext (System.Boolean forceNewContext) [0x00070] in <7071b8cb3c53472e8633e3e37f5e18d5>:0 
  at Duplicati.Library.Backend.SharePointBackend+<doList>d__28.MoveNext () [0x00032] in <7071b8cb3c53472e8633e3e37f5e18d5>:0 
  at System.Collections.Generic.List`1[T]..ctor (System.Collections.Generic.IEnumerable`1[T] collection) [0x0008b] in <8f2c484307284b51944a1a13a14c0266>:0 
  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x00011] in <63992662b765477a898ef49cdcc99ee2>:0 
  at Duplicati.Library.Main.BackendManager.DoList (Duplicati.Library.Main.BackendManager+FileEntryItem item) [0x0001b] in <df9891e588c04bd199c53e6d7539922f>:0 
  at Duplicati.Library.Main.BackendManager.ThreadRun () [0x000ee] in <df9891e588c04bd199c53e6d7539922f>:0 

When browsing to the mentioned URL, I can see the files that have been uploaded.

Running 2.0.2.13_canary_2017-11-22.

Yes, I have seen those errors on my Debian-OD4B backup, too.

This thread got some acitivity, fun! :slight_smile: Or, I guess it would have been better if it just worked, but now the problem may get some attention…

@rickard: If it fails on Linux, but works on Windows, my guess is that is some issue with Mono. Make sure you are using the Xamarin provided version (5.x) and not the old Debian provided version (3.x).

If possible, you can also try the latest build (2.0.2.15_experimental, should be available right now), and see if some of the general updates fixes the problem.

I have a bit of trouble debugging the issue, so I hope you can try various things to find out when it breaks. I have a test account for OD4B, but I can’t start throwing multiple GB at it to see if it breaks.

Hello @kenkendk!
Good call with the Mono library, I was running with the built in Debian default (4.6.2.7) and changed to Xamarin (5.4.1.6). However, this did not fix the issue… :confused:

I also installed the experimental Duplicati (2.0.2.15), and that did not change anything either. The behaviour is still the same, with timeouts during verification, and sometimes complete authentication errors.

Hello to everyone! I have the same issue. I don’t have any problem on Windows machines, but have on CentOS.

The backup created successfully, but I see error on test and after my attempt to restore a file.

Error with restore:
2020-07-17 17:41:36 +03 - [Error-Duplicati.Library.Main.Operation.RestoreHandler-PatchingFailed]: Failed to patch with remote file: “duplicati-baaa5cd4cbb594155a9a37bb12124a61b.dblock.zip.aes”, message: Cannot send a content-body with this verb-type.

Detail error with test:
17 Jul 2020 г., 17:53: Failed to process file duplicati-bb09685ac5fa9412e871208b1e44b5345.dblock.zip.aes
{“ClassName”:“System.Net.WebException”,“Message”:“Cannot send a content-body with this verb-type.”,“Data”:null,“InnerException”:{“ClassName”:“System.Net.ProtocolViolationException”,“Message”:“Cannot send a content-body with this verb-type.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at System.Net.HttpWebRequest.MyGetRequestStreamAsync (System.Threading.CancellationToken cancellationToken) [0x00070] in :0 \n at System.Net.HttpWebRequest.RunWithTimeout[T] (System.Func2[T,TResult] func) [0x0000d] in <a1451af8039c4226a11884e194023e35>:0 \n at System.Net.HttpWebRequest.GetRequestStreamAsync () [0x00000] in <a1451af8039c4226a11884e194023e35>:0 \n at System.Net.HttpWebRequest.GetRequestStream () [0x00000] in <a1451af8039c4226a11884e194023e35>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233079,"Source":"System"},"HelpURL":null,"StackTraceString":" at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <8f1de655bd1240739a78684d845cecc8>:0 \n at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x000b0] in <8f1de655bd1240739a78684d845cecc8>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2146233079,"Source":"Duplicati.Library.Main"} 17 Jul 2020 г., 17:52: Failed to process file duplicati-ib983cbbcbe774570a3691a6f35c35637.dindex.zip.aes {"ClassName":"System.Net.WebException","Message":"Cannot send a content-body with this verb-type.","Data":null,"InnerException":{"ClassName":"System.Net.ProtocolViolationException","Message":"Cannot send a content-body with this verb-type.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at System.Net.HttpWebRequest.MyGetRequestStreamAsync (System.Threading.CancellationToken cancellationToken) [0x00070] in <a1451af8039c4226a11884e194023e35>:0 \n at System.Net.HttpWebRequest.RunWithTimeout[T] (System.Func2[T,TResult] func) [0x0000d] in :0 \n at System.Net.HttpWebRequest.GetRequestStreamAsync () [0x00000] in :0 \n at System.Net.HttpWebRequest.GetRequestStream () [0x00000] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“System”},“HelpURL”:null,“StackTraceString”:” at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <8f1de655bd1240739a78684d845cecc8>:0 \n at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x000b0] in <8f1de655bd1240739a78684d845cecc8>:0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“Duplicati.Library.Main”}
17 Jul 2020 г., 17:51: Failed to process file duplicati-20200717T125853Z.dlist.zip.aes
{“ClassName”:“System.Net.WebException”,“Message”:“Cannot send a content-body with this verb-type.”,“Data”:null,“InnerException”:{“ClassName”:“System.Net.ProtocolViolationException”,“Message”:“Cannot send a content-body with this verb-type.”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:” at System.Net.HttpWebRequest.MyGetRequestStreamAsync (System.Threading.CancellationToken cancellationToken) [0x00070] in :0 \n at System.Net.HttpWebRequest.RunWithTimeout[T] (System.Func`2[T,TResult] func) [0x0000d] in :0 \n at System.Net.HttpWebRequest.GetRequestStreamAsync () [0x00000] in :0 \n at System.Net.HttpWebRequest.GetRequestStream () [0x00000] in :0 “,“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“System”},“HelpURL”:null,“StackTraceString”:” at Duplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile.get_TempFile () [0x00008] in <8f1de655bd1240739a78684d845cecc8>:0 \n at Duplicati.Library.Main.Operation.TestHandler.DoRun (System.Int64 samples, Duplicati.Library.Main.Database.LocalTestDatabase db, Duplicati.Library.Main.BackendManager backend) [0x000b0] in <8f1de655bd1240739a78684d845cecc8>:0 ",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:null,“HResult”:-2146233079,“Source”:“Duplicati.Library.Main”}

I use 2.0.5.1_beta_2020-01-18.

I don’t know why OS makes a difference, but you can read the below for notes and workarounds.

Ubuntu 16.04.6 LTS od4b Cannot send a content-body with this verb-type

In this July 11 error log, errors on verifications are similar to restore errors, in that both download.
No report from the other topic on whether the workarounds worked, but you can try them directly.