There are no downloads until after backup, and uploads would be limited to Options screen configuration for Remote volume size. Do you have a custom one that’s not the default 50 MB (still fishing for clues)?
Alternatively, are you talking about reading through a large source file, e.g. as seen on the home screen?
You can see live log About → Show log → Live → Information to see if a put starts but doesn’t complete. Alternatively set up a log-file at log-file-log-level=information or retry level (or higher, if a source problem).
For anyone getting occasional issues, you can check on a success to see whether the retries saved you.
“RetryAttempts” under Complete log in the usual job log spot. If you get some, maybe occasional error means you need more. There are lots of controls, including number-of-retries which is the easiest one…
I assume that this is the “full remote verify”, meaning a test with all. Is it full-remote-verification too?
Probably have to kill the process. Doing so while stuck in downloads should be safer than in a backup.
That didn’t run very long, correct? If you can get the same problem again, that would mean something.
You might as well also go to screen 5 (not 3) Advanced options and see if the following changes things:
--jottacloud-threads (Integer): Number of threads for restore operations.
Number of threads for restore operations. In some cases the download rate
is limited to 18.5 Mbps per stream. Use multiple threads to increase
throughput.
* default value: 4
Above is from an enhancement in 2.0.4.16_canary_2019-03-28 that has not yet gotten into the manual.
Try setting the option value to 1, as that looks like it will disable this rather unique feature. Does it help?
I think the issue is that there is some kind of rate limiting on Jottacloud. If you try to download a large amount of data in one go, it prevents you from doing it.
This is a major problem because to verify a large set you need to download all of it. In fact sometimes when compacting the set this happens too. I think I’ll look for a better cloud storage provider.
Firstly, when it looks like it has failed it hasn’t necessarily. The issue is that the Duplicati web UI stops updating in some circumstances. In particular, the command output seems to have some kind of line limit.
The live log view also doesn’t update after the machine sleeps and wakes up again. Refreshing the live log page makes it work again.
Secondly, Jottacloud has some kind of limit on the use of the CLI token. It’s not entirely clear what it is. I run Duplicati on two machines, and occasionally under some combination of factors I will get an authentication error. The token continues to work if I wait a while and try again, but of course whatever action Duplicati was taking fails.
Jottacloud is rather slow. I suppose it’s to be expected given the low cost, but it is faster via the web UI, or so it seems. It might be rate limited after you transfer a lot of traffic, I haven’t determined yet. Still, the only thing comparable for the cost and 4-5TB of data is iDrive S3, and their website is not all that clear about performance limits. Plus they don’t have a web UI or sharing features, if those are of use to you.
I an a full remote verify overnight with Jottacloud backend, and eventually started getting this error message:
Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
{"ClassName":"System.NullReferenceException","Message":"Object reference not set to an instance of an object.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":" at Duplicati.Library.Main.BackendManager.ThreadRun()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nThreadRun\nDuplicati.Library.Main, Version=2.0.7.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.BackendManager\nVoid ThreadRun()","HResult":-2147467261,"Source":"Duplicati.Library.Main","WatsonBuckets":null}
I had to generate a new OAth token before the Jottacloud backend could connect again.
There appears to be some unpublished limit on how much a token can be used. It’s less than 2TB, but I don’t know exactly how much.
a cursory read of the provider site leads to this: Reduced upload speed | Jottacloud Help Center
so above 5TB you will get reduced speed.
If Duplicati don’t know about it, it will fail because the cloud don’t answer fast enough. Try throttling it, there are advanced options for that (throttling-download and upload); there is no provision for modulating the throttling according to the backend size, though.
This is not a specific problem of your provider, OneDrive (for example) do something similar even for tiny test accounts of 5GB, see a current thread on this forum.
My account has 4.5TB on it, so it should not be throttled. It was downloading files quite quickly until it errored.
As I say, re-generating the OAuth key made it start working normally again. I’ll see if I can contact Jottacloud, maybe they can explain what the limits are.
I tried it with a test account and it seems that the cloud provider does not really want access with third party application, only with their own client, so support may be hard to get.