The remote server returned an error: (403) Forbidden with google drive

Support
1

Hi,
I noticed that my duplicati (2.0.2.1_beta_2017-08-01 on Win7) has most probably problem with deleting files on google drive. The whole backup goes OK, but when deleting unnecessary files it take several minutes and then ends up with the error.

Is there any way how to fix this?

Thanks,
Jakub

System.Net.WebException: The remote server returned an error: (403) Forbidden.
   at Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction)
   at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact)
   at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
2

That error means that it fails to download the file. Can you look in the log and see if there are other messages that could explain why it would fail to download a file?

3

Hi,
I tried again and got this in the live log on Info level (below). I checked the file exists on GD and also was able to download it manually to my desktop. Is there any action to rebuild the whole backup? Would that resolve the problem?

Thanks,
Jakub

first problem:

Sep 21, 2017 8:48 AM: Operation Get with file duplicati-be76096183e344943ad270da33b1e4690.dblock.zip attempt 1 of 5 failed with message: The remote server returned an error: (403) Forbidden.
{"ClassName":"System.Net.WebException","Message":"The remote server returned an error: (403) Forbidden.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"   at Duplicati.Library.Utility.AsyncHttpRequest.AsyncWrapper.GetResponseOrStream()\r\n   at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse()\r\n   at Duplicati.Library.Backend.GoogleDrive.GoogleDrive.Get(String remotename, Stream stream)\r\n   at Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)\r\n   at Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)\r\n   at Duplicati.Library.Main.BackendManager.ThreadRun()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nGetResponseOrStream\nDuplicati.Library.Utility, Version=2.0.2.1, Culture=neutral, PublicKeyToken=8bfe994a39631a7b\nDuplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper\nSystem.Object GetResponseOrStream()","HResult":-2146233079,"Source":"Duplicati.Library.Utility","WatsonBuckets":null}

then it re-tried few times:

Sep 21, 2017 8:51 AM: Operation Get with file duplicati-ba26dc6dabb4e4f1d87722f72388c8d98.dblock.zip attempt 1 of 5 failed with message: Thread was being aborted.
Sep 21, 2017 8:51 AM: Backend event: Get - Started: duplicati-ba26dc6dabb4e4f1d87722f72388c8d98.dblock.zip (49.97 MB)
Sep 21, 2017 8:51 AM: Backend event: Get - Failed: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:51 AM: Operation Get with file duplicati-be76096183e344943ad270da33b1e4690.dblock.zip attempt 5 of 5 failed with message: The remote server returned an error: (403) Forbidden.
Sep 21, 2017 8:50 AM: Backend event: Get - Started: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:50 AM: Backend event: Get - Retrying: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:50 AM: Operation Get with file duplicati-be76096183e344943ad270da33b1e4690.dblock.zip attempt 4 of 5 failed with message: The remote server returned an error: (403) Forbidden.
Sep 21, 2017 8:50 AM: Backend event: Get - Started: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:49 AM: Backend event: Get - Retrying: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:49 AM: Operation Get with file duplicati-be76096183e344943ad270da33b1e4690.dblock.zip attempt 3 of 5 failed with message: The remote server returned an error: (403) Forbidden.
Sep 21, 2017 8:49 AM: Backend event: Get - Started: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:49 AM: Backend event: Get - Retrying: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:49 AM: Operation Get with file duplicati-be76096183e344943ad270da33b1e4690.dblock.zip attempt 2 of 5 failed with message: The remote server returned an error: (403) Forbidden.
Sep 21, 2017 8:48 AM: Backend event: Get - Started: duplicati-be76096183e344943ad270da33b1e4690.dblock.zip (49.93 MB)
Sep 21, 2017 8:48 AM: Backend event: Get - Retrying: duplicati-

and final fatal error:

Sep 21, 2017 8:51 AM: Fatal error
{"ClassName":"System.Net.WebException","Message":"The remote server returned an error: (403) Forbidden.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"   at Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()\r\n   at Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction)\r\n   at Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact)\r\n   at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)\r\n   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)\r\n   at Duplicati.Library.Main.Controller.<>c__DisplayClass16_0.<Backup>b__0(BackupResults result)\r\n   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)\r\n   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)\r\n   at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8\nget_TempFile\nDuplicati.Library.Main, Version=2.0.2.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.AsyncDownloader+AsyncDownloaderEnumerator+AsyncDownloadedFile\nDuplicati.Library.Utility.TempFile get_TempFile()","HResult":-2146233079,"Source":"Duplicati.Library.Main","WatsonBuckets":null}
4

Did you manually touch the file?

If you create the AuthID with the ā€œGoogle Driveā€ setting, the permissions are set such that Duplicati can only touch ā€œown filesā€. This is a security mechanism that protects agains Duplicati throwing a ā€œfitā€ and wiping your entire Google Drive (extremely unlikely to happen, but ā€¦).

If you ā€œtouchā€ the files (rename, move, etc) Google makes sure that ā€œyouā€ own the files now, and prevents Duplicati from touching them.

Is it possible something like this has happened?

5

Would opening the file online or downloading the file from GD cause change of ownership? What shall I do to get rid of the error? Is there way to change the ownership on GD? or shall I manually delete the file?

Thanks,
Jakub

6

I edited your post to improve the formating. (Just added ~~~ before and after the outputs you pasted, see here for details).

7

I donā€™t know; that is for Google to decide.

The easy fix is to create a new AuthID using the Google Docs option, which grants Duplicati full access to all your files.

The slightly harder fix, is to download the file, delete it from Drive, then use the backend tool to re-upload it:

Duplicati.CommandLine.BackendTool.exe PUT googledrive://folder?authid=<original auth id> <filename>

A third option is to just delete it, and then use purge-broken-files to get the backup in shape again.

8

Sorry to revive an old thread but Iā€™m having this issue and canā€™t seem to resolve it. I even made a new backup using a full google drive AuthID from the start and by the second backup this happened again. I never access the files directly, although a different folder in the GDrive is used for rclone. Do you have any suggestions?

Errors: [
    Failed to process file duplicati-20180630T151905Z.dlist.zip.aes => The remote server returned an error: (403) Forbidden.,
    Failed to process file duplicati-i83a0e6ad1869400ea57c046ef347ce26.dindex.zip.aes => The remote server returned an error: (403) Forbidden.
]
9

Am I right to assume the rclone process isnā€™t having any issues?

10

It seems to complete but with errors as mentioned. Verify gives the same errors. As far as I can see, I can restore the data as expected but Iā€™m not sure if the errors indicate certain files will not be able to be restored. Donā€™t want to learn that when itā€™s too late.

11

Good thinking!

Iā€™m not aware of any other users having issues with Google Drive at the moment so for now Iā€™m assuming this is something specific to your setup / drive / AuthID.

The two error files are a dlist (list of what was backed up on that run) and a dindex (list of what file blocks are stored in what dblock.zip.aes files).

Neither file contains actual backup data and can be regenerated from the local sqlite databse, so I donā€™t think youā€™re at risk of not being able to restore. The worst case scenario I can think of is if you have to restore from direct from the destination (as in your local sqlite file isnā€™t available) it will take a bit longer because it will have to regenerate the contents of those two files.

Is it always the same two files giving the error? If so, are you able to move those files out of the Google Drive destination folder and try a repair to get them re-generated? (Be sure to move, not delete, so we can put them back if it doesnā€™t work.)

As with the original posters issue Iā€™m wondering if the ownership of those files has somehow changed causing Duplicati to no longer have permission to read them.

Oh - and for your new AuthID did you make it for Google Drive (Duplicati can only see itā€™s backups) again or for Google Docs (Duplicati can see ALL your files, including these two that likely changed ownership.)

12

Thanks for the detailed reply. It seems to be a different file each run. The new auth ID I tried out was a full one for Google Drive (full access).

I made another new full backup, even the first run failed. This was with a limited login.

estResults:
MainOperation: Test
Verifications: [
Key: duplicati-20180707T131922Z.dlist.zip.aes
Value: [
Key: Error
Value: The remote server returned an error: (403) Forbidden.
],
Key: duplicati-ifbcb8011a9b64b6e981020dbd7efa8ac.dindex.zip.aes
Value: [],
Key: duplicati-bbf50b20d2ae24fb5b49d696222a8d2b4.dblock.zip.aes
Value: [
Key: Error
Value: The remote server returned an error: (403) Forbidden.
]
]

Errors: [
Failed to process file duplicati-20180707T131922Z.dlist.zip.aes => The remote server returned an error: (403) Forbidden.,
Failed to process file duplicati-bbf50b20d2ae24fb5b49d696222a8d2b4.dblock.zip.aes => The remote server returned an error: (403) Forbidden.

13

I setup another clean backup this time using the Google Drive (full Access) auth ID, and again had an error with the initial backup.

Errors: [
Failed to process file duplicati-20180707T145415Z.dlist.zip.aes => The remote server returned an error: (403) Forbidden.
]

14

How did you set up the full access auth ID? Iā€™m trying to make a test job and can only get limited access with the Google Drive storage type.

15

Good question, I assumed I was doing it right but now Iā€™m doubting that. I just used the normal o-auth handler (https://duplicati-oauth-handler.appspot.com/ ) and selected ā€˜Google Drive (full access) loginā€™ instead of the default limited.

16

Odd, I donā€™t see full access as an option anywhere - Iā€™m sure Iā€™m doing something wrong, but I donā€™t know what.

image
image.png670Ɨ734 43.1 KB

17

You can just go to the link in my last post directly(itā€™s the same base URL in your screenshot) and click the full access button then copy and paste it into the Auth ID field on the page.

18

Heh, thanks - my brain isnā€™t working today I guess.

I just set up a test job using Google Drive (full access) and ran it twice without issue. Of course it was only 2 files, so maybe that has something to do with it.

Itā€™s possible the file trying to be access is the one that Duplicati just uploaded and Google hasnā€™t ā€œturned-it-aroundā€ yet to make it downloadableā€¦

Does the error go away if you set --backup-test-samples=0? This basically tells Duplicati not to do a post-backup test of any files - so itā€™s not really a solution, but could help narrow down the source of the issue.

--backup-test-samples
After a backup is completed, some files are selected for verification on the remote backend. Use this option to change how many. If this value is set to 0 or the option --no-backend-verification is set, no remote files are verified
Default value: ā€œ1ā€

19

Thanks for the suggestion. Yes, that eliminates the error but as you noted, itā€™s not ideal to skip that step. If I try to verify the backup manually it still produces the error of course. Does knowing this lead you to any further thoughts? Thanks.

20

If you try the manual test a few hours after the last backup finished and youā€™re still getting the error, the perhaps the issue with the permissions on the files themselves.

I think the first thing to do is to test a restore of a file or two - just to make sure that error doesnā€™t happen if youā€™re needing to restore anything. If it DOES happen, thereā€™s likely a general authorization problem at Googles end.

We have seen one or two cases where somehow the files Duplicati uploads get their ownership changed causing Duplicati to no longer be able to access them, however I donā€™t recall any specific resolution for he issue being found. I think the user who reported it had started a fresh backup and the error never came back.

1 Like