Setting up OneDrive (personal)

For three shots in the dark.

  1. Try it from the Duplicati OAuth Handler directly instead of on Destination screen, and carefully copy.

  2. Go to https://account.live.com/consent/Manage and look over the Duplicati entry or entries, or if you feel more comfortable navigating, go to your account’s Privacy settings to find " Apps and services". Probably Duplicati would have below, but probably there was a consent page shown at first creation.

  • Access OneDrive files

  • Maintain access to data you have given Duplicati access to

  1. Invalid authid in query which you were seeing is interesting because AuthID is a Duplicati term, which from source looks superficially (but I also just tested it) like it means your AuthID lacked a colon. Don’t post it here, but do you see any colon in the rest of the random-appearing numbers and letters?

Or maybe this will remain a mystery until the person who got it going says what it actually took to get it going.

1 Like

Did that multiple times before, no luck.

OK, this was productive. I removed 2 Duplicati entries and then generated a new AuthID in Duplicati settings. Now I get a new (albeit shorter) error message:

Failed to connect: Failed to authorize using the OAuth service: Server error. If the problem persists, try generating a new authid token from: Duplicati OAuth Handler

I’ve since generated a new one and still get the same error message. Any ideas?

And yes, my AuthID code has a colon :slight_smile:

FWIW, I created a matching support thread for this on Microsoft Answers.

The Microsoft Answers thread didn’t mention 2FA (if you’re still using it). Although I’m not sure how that looks in Graph API, do you recall your previous interaction? It would seem like it would have to be somewhere around a Microsoft screen. Duplicati doesn’t handle any 2FA details, and I doubt it even knows that you want to use 2FA, however I’d sort of hope Microsoft doesn’t let it just slide in, if your account is configured for it to be required…

FWIW, I’m having the EXACT same problem as @jdrch, right down to the symptom that the email that shows up initially is my enterprise email, not my consumer onedrive MSA email.

I don’t have any answers but happy to test out suggestions.

Success!
When you are on the Duplicati backup destination screen, it appears that Duplicati is adding an extra (redundant?) authid header in the advanced options. If you delete this advanced option, then generate a new authid, then click test, it seems to now work.

1 Like

You’re gonna die laughing, but the reverse just worked for me. Brilliant idea. Full start to finish steps, for anyone else with the problem:

  • Go to https://account.live.com/consent/Manage and search the page for “Duplicati.”
  • Click on each Duplicati entry from the above page
  • In the page that loads afterwards, click Remove. Repeat this until all Duplicati entries on the Manage page are gone
  • In the Duplicati web UI, under Backup Destination, select Microsoft OneDrive V2
  • In path, enter the folder path of the backup in the format Folder/ChildFolder. There are no symbols, indicators, or characters necessary for the root directory.
  • Click the AuthID link and sign into your Microsoft Account
  • Grant Dupilcati permissions in the dialog that pops up. This should automatically populate the AuthID field
  • Expand Advanced Options
  • Expand Add advanced option drop down menu
  • Under Microsoft OneDrive V2, select authid: the authorization code
  • In the authid field that opens up above the Add advanced option menu, copy and enter the AuthID generated from the link. Ensure the same AuthID is in the AuthID field under the Backup destination heading.
  • Click Test Connection. You’ll get an error message about “duplicate authid”, but ignore it. The test should be successful.
  • Set up the rest of the backup as the documentation specifies
  • Run the backup and check the live verbose log to ensure there isn’t any connection error.
  • After that backup has completed successfully, go back to the Advanced Options setting and remove the authid entry.
  • Test the connection again. It should pass with no errors.
  • Click Next until you can save the config.
  • Click Save to save the config.
1 Like

Wow… The AuthID that is entered in the dialog for setting up the backend is encoded into a “connection url”, so it will look something like onedrivev2://folder/subfolder?authid=abc:123.

The idea is that a service wanting to support Duplicati backups can have a button for “copy connection url” that you can paste in and have it all configured.

It should work such that the query parameters from the url override the advanced options, meaning that the override you put into the Advanced options should be ignored, as the url parameters take precedence.

I was guessing that it was an encoding issue, but since it works after you remove the advanced options, it must be something else.

It could be that it is simply the OAuth process that is unstable. I often see in the logs that OAuth fails for all requests towards a specific provider (e.g. Microsoft) for a period, and then starts working again with no change on my side.

1 Like

Hi, may be the same problem:
https://forum.duplicati.com/t/problems-with-sign-in-a-command-line-export/5951/5

You are the man!
Similar to jdrch, I went to https://account.live.com/consent/Manage, removed all Duplicati entries. And then I did what you suggested and it works. Thanks!

1 Like

Hello Guys,

i am writing you from germany. So please excuse me for my bad english. :wink:

I made all the points exactly as jdrch wrote it. But I still get an authorization error. I use duplicati on a Synology DS218+ NAS.

What information do you need to help me?

Greetings from Germany!
Nico

An old thread but…
I have family O365. I see my onedrive (1T per peson), and can push things to it!
I have configured duplicati to use onedrive. I added a folder on onedrive, then added that name into the configuration (step 2). Got the Auth code and hit test.
it returned to say that the folder did not exist, did I want to add it, yes I do!
The backup failed (trailing slash on the name I think)
So I started again, only this time I did not create a folder, but used a different name at step 2
The test worked, the backup worked!
But where is my backup folder? I can not see it in onedrive!
If I re-run the backup, it runs without complaint!
Any ideas?

I have been using Duplicati for some time with various backup destination types, and so far I have been satisfied with it. I am considering whether OneDrive would be an option to add for certain backups going forward, and I read this thread and other related information (e.g. How We Get Along With OAuth - Duplicati 2 User's Manual).

After reading this I am left with a security concern that I am curious to hear opinions about. I think I sort-of understand that Duplicati’s OAuth service has been set up in such a way that the risk is limited if an attacker would compromise the service’s database. But am I wrong in assuming that for this system to be secure, you must place 100% trust in the service itself? It seems to me that it would be very straightforward for someone with access to the service to store all the information required to access OneDrive without the need for the local instance of Duplicati – it would probably just need to store the authid it generates. Even if one trusts the service in principle, wouldn’t an attacker with access to the service have the power to modify it such that authids are stored locally, thereby gaining full access to all OneDrive accounts? Or do I misunderstand something about how this works? (It certainly isn’t my expertise.)

Also, I think I sort-of understand why it isn’t smart to store all information in the local Duplicati instance (all information being whatever the service stores at the moment), but on the other hand I also don’t really understand why it is not an option. If the information is compromised an attacker could have full access to my backups. However, this doesn’t seem different from e.g. a WebDAV destination or any other destination that can be accessed directly by Duplicati (at least not when the OneDrive account is only used for backups). In addition, if attackers can get this information from my local Duplicati installation this almost certainly means my entire system is compromised, which would give them access to the original data anyway (no need to access the backup). So why isn’t this a possibility, to avoid having to trust an external service about which I have no knowledge when it comes to security?

I think I am probably missing something here. Could anyone explain what I am missing?

Yes, that is correct. It is designed such that a leak or access to the stored data in the service does not compromise any tokens.

Yes, that is what tokens do; they allow you to authenticate as the user they were issued for. It is no different than hijacking session tokens in a browser.

That is correct. The authid is essentially a “password” that decrypts the stored token and uses it to authenticate with the destination service.

Sadly, this is required due to the way OAuth is designed. For OAuth to work, the service needs to store a token for each user. In other systems, this is designed to be guarded by a login for the service, but since we do not have users in that sense, the authid covers the role of authenticating and unlocking the secret token for a short a period as possible

Yes. That is why the service is hosted on Google App Engine, which I count on as being a trusted hosting service that prioritizes security. See also my answer above.

Again, this is due to how OAuth is designed. Each application that uses access via OAuth needs to store an “application secret” that authenticates each request as being emitted by the application. It then serves as a sort of kill-switch that the OAuth owner (e.g. OneDrive) can use to immediately cut off an application that has defects. This limits the impact of a faulty third-party but also requires that the secret is stored outside the application (i.e. on a closed server).

That is sadly a good observation about the logic behind OAuth. And most services (including OneDrive) does not allow you to access the API without authenticating via OAuth (i.e.: you cannot just send your username and password).

Some services have started allowing Personal Access Token (PAT), which is essentially a random extra password for your account with selectable privileges, but this is not common across services yet. If this option is allowed for a backend, it should be used to avoid roundtrips to the OAuth server.

For Duplicati, you can actually implement your own OAuth server and avoid a trusted third party. The source code of the auth-handler is freely available. It runs on Google App Engine but can quickly be adapted to run without the GAE datastore.

However, before you can use OAuth you need to register with the provider (i.e. OneDrive) and create application. The you register the allowed callbacks and gather the application secret, and you can run your own trusted OAuth solution.

I hope this is covered?

Thanks for the explanation. It does clarify a few things for me, though I think I still need to read up on a few things to really understand. I guess that the main thing that I don’t understand is why Duplicati itself doesn’t contain the functionality offered by the auth-handler service. The option to run a custom (trusted) version is interesting and would solve the issue, but it may go too far for me at this time.

I suppose that for me, for now, it means that backing up to OneDrive is not a viable option without requiring a lot more effort, meaning that it is most likely easier to find a different cloud solution. I will have to weigh some pros and cons against each other again.

That is due to the design of the OAuth protocol. It is not a great fit for solutions that does not have a web-based user interface (i.e. anything that is not a web page).

It is trivial to write the OAuth handler in C# or just bundle the Python version with Duplicati. But for it to work it needs the “application secret”, which is a secret value obtained when signing up as a third-party provider. Any communication with OneDrive needs the “application secret” or it will be rejected. The provider (Microsoft in this case) uses this to revoke access to applications that are not compliant with their ToS.

If I were to ship the “application secret” with Duplicati, it would be publicly visible and hence not a secret. This would allow anyone to pretend to be Duplicati and impersonate requests. Once this happened, Microsoft would revoke the token, causing all Duplicati users to loose access.

The reason why I am not shipping the OAuth handler with Duplicati is because it is useless without an “application secret”.

To get that part you sign up as a third-party application with Microsoft, approve the ToS, set up an application, and then enter the “application secret” locally into the service. For most users this setup would be too much of a hassle.

Aaah ok, I think I see what you mean now…

It would appear that I either need to get over my ‘trust issues’ or stay with a different storage provider then. Thank you for answering my questions!

There are still numerous Storage Providers not seen on Duplicati OAuth Handler screen.

Not all OAuth server uses are the same, for example Google Drive (limited) login
scope limits Duplicati to files it made. It can’t see others. It lowers risk, but is confusing…

An Illustrated Guide to OAuth and OpenID Connect first 11 minutes shows some basics.

@kenkendk
This thread was a real eye-opener for me. I appreciate the transparency on security in this thread, and IMO this information is important and needs to be much more widely available than it is buried here.
The fact that Duplicati is not 100% E2EE/ZK – when OAuth is involved – was a total surprise to me and is a big deal from a security point of view. I think it needs to be spelled out very clearly in the core documentation.
Yes the Duplicati documentation doesn’t say that Duplicati is E2EE/ZK (as far as I can tell), but anyone reading the documentation would assume it is, given that Duplicati is positioned as software running on the user’s computer. Nowhere is there any (obvious enough) mention that it doesn’t run entirely on the user’s computer.
This also raises the important question: are there any other components of the backup functionality (other than the OAuth handler mentioned in this thread) that runs in the cloud (i.e., anywhere else than on the user’s computer)?
I love Duplicati and have found it very functional and reliable, but I have to admit that my love is a bit less enthusiastic now that I know it’s not E2EE/ZK - especially since I only found that out because I just happened to read this forum thread.
.
Can I make some “wish list”-type suggestions for changes to the Duplicati.com website content and the help pages (readthedocs.io)? (My apologies if some of this exists but I didn’t find it.)

  1. The “How we get along with OAuth” page talks about a “service” that handles OAuth. But reading this, a user would assume (at least I did) that this service was running on my own computer, like the rest of the Duplicati software, which is in fact running as a service. It’s critical from a security point of view to make it very clear that the OAuth handler service is actually running in the cloud.
  2. There should be a new help page that describes Duplicati’s security and security model and all issues like the above.
    • The new page would describe that everything runs on the user’s computer, except for X, Y, and Z which run in the cloud. (E.g., X = OAuth for One Drive and some other services)
    • It would list all the storage providers that depend on something running in the cloud, and for each one would list what is the thing running in the cloud. (E.g., OneDrive: OAuth handler service running in Google App Engine)
    • It would describe all the security measures you’ve taken to protect the cloud OAuth handler service, and would list the risks that you can’t do anything about (residual risks).
    • It would explicitly describe how Duplicati is E2EE/ZK unless the storage provider needs any of those X, Y, and Z things. (Or maybe some of the Y and Z don’t affect E2EE/ZK-ness or any other security aspect?)
    • It would point to the “How we get along with OAuth” page, and to the “Duplicati OAuth Handler” page.
  3. There should be a short blurb on the Duplicati.com main page that talks about security and links to the new help page I suggested. The existing “Strong encryption” blurb could be replaced with something like “Strong security” and this new blub.

I think this is not a correct interpretation. Duplicati performs all encryption client-side and is treating any remote storage service as a “bucket of encrypted blobs”. In my view that is a 100% compliance to TNO or EE2E/ZK.

No. The system designed is made so there is minimal need for service maintenance by me. The OAuth service was added because all the storage providers want to prevent users storing their real account credentials in third-party applications, and because the OAuth design is not ideal for a client-only application such as Duplicati.

There are two other pieces to the Duplicati service eco-system: logging and updates. Each of these run separately and have no impact on backups. Both can be disabled completely.

Just to make it clear: the OAuth service is an unwanted necessity that ensures that your real username/password is newer exposed to Duplicati. The OAuth setup essentially swaps your username/password with a token that has limited permissions. This ensures that you can revoke any access to Duplicati and that the service provider can disable Duplicati if they deem it necessary.

The need for the OAuth service arises because the authentication returns two tokens: access token and refresh token. The access token can be used directly instead of username/password but expires “quickly” (typically 4 hours). The refresh token is used to obtain a new access token, but requires the presence of the application secret (mentioned previously in this thread).

This has two important implications:

  • Anyone with a valid access token can access the same files as Duplicati
  • If the OAuth service breaks, backups cannot start

To limit the possibility of the first implication, everything stored in the OAuth service is encrypted with the AuthID.

The second implication is not mitigated. Once PATs are more generally in use, we can retire the OAuth service.

To sum up: the OAuth service does not see any files or operations performed on the storage. A compromised OAuth token allows an attacker to perform the same remote file operations as Duplicati (list, put, get, delete). Any operations are executed by the client; the OAuth service does not interact with the storage in any way.

We wrote the “How we get along with OAuth” page with the exact intention of describing the security implications, as the rest of Duplicati works without any external service.

If you have specific changes that you request, you can make a pull request for the main site.

Settings in Duplicati mentions (probably less sensitive) updates and usage reports (even prints URL).

Any OAuth issue is only relevant to OAuth. In AuthID setup you can see URL and read privacy policy:

Not everyone pays attention to the URL they are at, or reads privacy policy (which does get technical).

Duplicati OAuth Service Privacy Policy =

This document describes what information the Duplicati OAuth service collects, how it is used, how and where it is stored, whom it is shared with, and how you can remove the stored information.

== How we use your data ==
The Duplicati OAuth service attempts to collect and store as little information as possible. When you connect to any of the supported OAuth providers, Duplicati obtains a long-term session key. This key, combined with a secret known only to the Duplicati OAuth service, can be exchanged for a short-term key, which can be used to access your account on the site that you have authenticated with.

== How we store your data ==
To avoid transmitting the long-term session key, we store the long-term session key. To prevent unwanted access to your account, we encrypt the long-term session key with a password before we store it. This password is given to you in the form of an “AuthID” string. It is not possible to recover the long-term session key without this password, meaning it is only possible to intercept this key during a login (and shortly after, as the key is briefly kept in cache). Due to the way OAuth is constructed, we do not have access to, and thus do not store, any personally identifiable information. This includes not storing username, email or password to the authenticating site.

== Where we store and handle your data ==
The data is stored on Google App Engine, using their scalable storage and server infrastructure. As part of using GAE we are storing log information that shows IP addresses and some request information. This data is stored for a short period, as the amount of data is quite large, and new data overwrites existing data.

== Who we share your data with ==
As part of the login process, we send the long-term session token to the provider who issued it (after it has been decrypted with the client-provided key). The resulting short-term session key is sent directly to the requesting client, and not stored on the server.

Apart from this exchange of secrets, we do not share your information with anyone. This is enforced since we do not store any information that we could share, except from the log data as described above. Please note that as the service is running on GAE, Google will also have access to the log data.

== How to remove the stored information ==
You can remove a stored “AuthID” at any time by providing the “AuthID” you want to remove to the removal page, listed on the website. Removal is performed immediately and irrevocably.

If you do not have the AuthID, we cannot remove the information, but since it is encrypted it is not accessible by anyone either.

To ensure that permissions are revoked, you can also use the site you are authenticating with, and revoke access by Duplicati. This will permanently invalidate all the stored (and encrypted) long-term session tokens on the Duplicati OAuth service.

Logins that have not been used for a long period of time are automatically removed.

I suppose OAuth service’s blurb could say “cloud service”. There’s probably an update coming soon.
The manual is a third-party project. Its maintainer seems to be short on time but invites pull requests.
Duplicati is a community effort, and is entirely limited by the volunteers who support it with their time.

Anybody who appreciates the benefits that sometimes only non-commercial works offer, please help.
Almost anybody can contribute their skills to some area (even forum), even if they’re not a developer.