Jottacloud Error 401 (Unauthorized)

ning42 · July 1, 2022, 4:13am

@shalmirane - anyway you can build a docker image agin with these ?

shalmirane · July 1, 2022, 7:02am

I’m not the author - just copy/pasted the proper command to fetch it

But pinged @simoncaron on github, who is.

waLIEN · July 8, 2022, 5:02pm

After 2 weeks+ of successful updates, it appears the OAUTH token finally expired:

Failed: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
Details: Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   at Duplicati.Library.OAuthHelper.CreateRequest(String url, String method, Boolean noAuthorization)
   at Duplicati.Library.JSONWebHelper.GetJSONData[T](String url, Action`1 setup, Action`1 setupbodyreq)
   at Duplicati.Library.Backend.JottacloudAuthHelper..ctor(String accessToken)
   at Duplicati.Library.Backend.Jottacloud..ctor(String url, Dictionary`2 options)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Duplicati.Library.DynamicLoader.BackendLoader.BackendLoaderSub.GetBackend(String url, Dictionary`2 options)
   at Duplicati.Library.Main.BackendManager..ctor(String backendurl, Options options, IBackendWriter statwriter, LocalDatabase database)
   at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

Log data:
2022-07-08 03:00:40 -07 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   at Duplicati.Library.OAuthHelper.CreateRequest(String url, String method, Boolean noAuthorization)
   at Duplicati.Library.JSONWebHelper.GetJSONData[T](String url, Action`1 setup, Action`1 setupbodyreq)
   at Duplicati.Library.Backend.JottacloudAuthHelper..ctor(String accessToken)
   at Duplicati.Library.Backend.Jottacloud..ctor(String url, Dictionary`2 options)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Duplicati.Library.DynamicLoader.BackendLoader.BackendLoaderSub.GetBackend(String url, Dictionary`2 options)
   at Duplicati.Library.Main.BackendManager..ctor(String backendurl, Options options, IBackendWriter statwriter, LocalDatabase database)
   at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()```

wmrch · July 10, 2022, 3:52am

Sorry, I’m a vanilla Duplicati-on-Windows user. I don’t quite see through it: Is it now possible to apply the fix with Duplicati on Windows, or is it only something you can do if you create Docker images yourself?

Is there any way to get my backups working again with Jottacloud?

FreaQ · July 12, 2022, 7:03am

If you use the latest canary version Jottacloud is working more or less.
There are problems with long running backups where the token expires.
But you can always create a new token and restart your backup until it finishes complete. This may take some patience.

FredericM · July 12, 2022, 5:53pm

I’ve compiled the changes from the PR “Disable automatic use of v2 authid for Jottacloud” of @albertony.
This generated 3 updated DLLs that I’ve replaced on my Duplicati instance. I’ve one long running backup (low upload speed limit by ISP) and the backup is now running for 27h+ where it generated an error in 8h or less before.

I’ve uploaded the DLLs below for the people who are interested.
I’m using Duplicati in a Debian and the DLLs were located at /usr/lib/duplicati.

WARNING: I don’t take any responsibility for any mistake or failure that might happen. Make sure to take a backup of the files before overwriting them. Or even better, make a backup/snapshot/… of the machine/container/… I’m running Duplicati inside an Linux container on Proxmox so I just took a snapshot before overwriting the files. Use at own risk!

OAuthFixJottacloud.zip (63.2 KB)

wmrch · July 16, 2022, 1:10pm

Thanks alot, after switching to Canary and setting up the token it seems to work for now.

FreaQ · July 22, 2022, 5:47am

I replaced the DLLs on my QNAP installation and ran several backups without problems.
I did not have any long running backups, but when executing a lot of backups successively in the past the token would also fail which did not happen this time.
So I am slightly positive that the changes done by @albertony are fixing the token problem.
Thank you.

waLIEN · July 26, 2022, 5:44pm

For those of you who have it working consistently, are you using the default value of “4” for " –asynchronous-concurrent-upload-limit"?

Duplicaterr · July 26, 2022, 6:16pm

Yes, didn’t change that value and it’s working without issues.

dukrat · July 29, 2022, 2:10pm

I also compiled albertony/duplicati at jottacloud-disable-v2-authid (github.com) and replaced the 3 DLLs on Windows and Linux. It did not completely eliminate the issue but it did allow the backups to run for many hours before failing when before it seemed to fail after less than an hour and only a few files.

I have a 2.1 million file 3.3 tb Windows backup that normally runs in 4 hours but was severely behind, it took 6 retries after switching to the patched DLLs and now completes again in 4 hours. I also have a 156 thousand file 21 gb Linux backup that runs in 10 minutes it would die about 2 minutes in before the patch and immediately finished after it.

I made no changes to asynchronous-concurrent-upload-limit

waLIEN · August 2, 2022, 2:49am

I’m on the latest canary build with the 3 new DLL files. It seems that my larger backups are still intermittently failing with the same OAUTH failure. Below are the logs – please let me know what else I can provide to help:

Failed: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
Details: Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud —> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
at Duplicati.Library.OAuthHelper.GetTokenResponseT
at Duplicati.Library.OAuthHelper.get_AccessToken()
— End of inner exception stack trace —
at Duplicati.Library.Main.BackendManager.WaitForEmpty(LocalDatabase db, IDbTransaction transation)
at Duplicati.Library.Main.Operation.CompactHandler.DoDelete(LocalDeleteDatabase db, BackendManager backend, IEnumerable1 deleteableVolumes, IDbTransaction& transaction) at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend) at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager) at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize) at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task) at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result) at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action1 method)

Log data:
2022-08-01 19:34:05 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:34:53 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:35:41 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:36:32 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:37:10 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:37:59 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:38:48 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:39:37 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:40:28 -07 - [Warning-Duplicati.Library.Main.BackendManager-BackendDisposeError]: Failed to dispose backend instance: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
System.NullReferenceException: Object reference not set to an instance of an object.
at Duplicati.Library.Main.BackendManager.ThreadRun()
2022-08-01 19:40:28 -07 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud —> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
at Duplicati.Library.OAuthHelper.GetTokenResponseT
at Duplicati.Library.OAuthHelper.get_AccessToken()
— End of inner exception stack trace —
at Duplicati.Library.Main.BackendManager.WaitForEmpty(LocalDatabase db, IDbTransaction transation)
at Duplicati.Library.Main.Operation.CompactHandler.DoDelete(LocalDeleteDatabase db, BackendManager backend, IEnumerable`1 deleteableVolumes, IDbTransaction& transaction)
at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)
at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBackend, Boolean forceCompact, BackendManager sharedManager)
at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)
at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()

Duplicaterr · August 2, 2022, 7:08pm

Since my last reply to you I’ve been constantly running a backup (have to rerun a 6TB set because I initially set the blocksize too low) and didn’t encounter a 500 error.

If you generate a new authid token - does it work again?

waLIEN · August 2, 2022, 9:41pm

If I generate a new AuthID token, it works again for a few weeks.

I did make one change several days prior, in that I removed the custom field: " –asynchronous-concurrent-upload-limit which was set to “1” for testing. Though, I don’t think this has an effect as it’s been working for a few days prior to failing.

SirTerrific · August 8, 2022, 2:51pm

Hi,
I’m still disconnected with a 500 error. I’m not able to backup my 4 TB anymore since June … What is your configurations ?
thanks.

jankkm · August 8, 2022, 5:09pm

Thank you so much! Uploads have been running non stop for a couple of days without any issue.

I used the 3 DLLs without a change to ‘asynchronous-concurrent-upload-limit’

waLIEN · August 19, 2022, 6:23pm

I have to set the following setting (default is 4) for it to work consistently:
" –asynchronous-concurrent-upload-limit which was set to “1”

waLIEN · August 25, 2022, 11:20pm

Does anyone with large libraries (4TB) with large incremental backups still run into the OAUTH errors? It seems like my smaller backups are fine, but the large ones fail regularly if there is a large incremental backup. This happens even with freshly generated OAUTH tokens. Here are the logs:

Failed: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud
Details: Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   at Duplicati.Library.OAuthHelper.CreateRequest(String url, String method, Boolean noAuthorization)
   at Duplicati.Library.JSONWebHelper.GetJSONData[T](String url, Action`1 setup, Action`1 setupbodyreq)
   at Duplicati.Library.Backend.JottacloudAuthHelper..ctor(String accessToken)
   at Duplicati.Library.Backend.Jottacloud..ctor(String url, Dictionary`2 options)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Duplicati.Library.DynamicLoader.BackendLoader.BackendLoaderSub.GetBackend(String url, Dictionary`2 options)
   at Duplicati.Library.Main.BackendManager..ctor(String backendurl, Options options, IBackendWriter statwriter, LocalDatabase database)
   at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

Log data:
2022-08-25 16:40:40 -07 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
Duplicati.Library.Interface.UserInformationException: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: https://duplicati-oauth-handler.appspot.com?type=jottacloud ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error.
   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()
   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()
   at Duplicati.Library.JSONWebHelper.GetResponse(AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.JSONWebHelper.ReadJSONResponse[T](AsyncHttpRequest req, Object requestdata)
   at Duplicati.Library.OAuthHelper.GetTokenResponse[T]()
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   --- End of inner exception stack trace ---
   at Duplicati.Library.OAuthHelper.get_AccessToken()
   at Duplicati.Library.OAuthHelper.CreateRequest(String url, String method, Boolean noAuthorization)
   at Duplicati.Library.JSONWebHelper.GetJSONData[T](String url, Action`1 setup, Action`1 setupbodyreq)
   at Duplicati.Library.Backend.JottacloudAuthHelper..ctor(String accessToken)
   at Duplicati.Library.Backend.Jottacloud..ctor(String url, Dictionary`2 options)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Duplicati.Library.DynamicLoader.BackendLoader.BackendLoaderSub.GetBackend(String url, Dictionary`2 options)
   at Duplicati.Library.Main.BackendManager..ctor(String backendurl, Options options, IBackendWriter statwriter, LocalDatabase database)
   at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__20.MoveNext()```

Dominic-Schaefer · August 26, 2022, 8:29am

I have exactly the same still.
All my smaller backups run through without issues.
My large one (<1TB but millions of files) fails every time with the Oauth expiring.

SulfurRacoon · August 29, 2022, 3:03pm

I have the same issue but with backups less than 1 TB, too.
Not only with backup jobs but also with database repair.
After several time the progress bar won’t grow any further and even after hours the job isn’t done. So I only can kill the process in the task manager. After this each time I need a new Oauth token, because the old one isn’t working any longer.