Uploading to OneDrive always breaks after ~20min

1

Hi all,

I’m using duplicati to back up ~250GB to OneDrive (personal). Setting everything up worked like a charm using this guide: Setting up OneDrive (personal)
I had to set the accept-any-ssl-certificate option to make things work though (I followed the SSL guides with no success).

Starting the backup works, however, after some time the backup breaks with the following error message:

Failed to authorize using the OAuth service: Error: TrustFailure (The authentication or decryption has failed.). If the problem persists, try generating a new authid token from: Duplicati OAuth Handler

It doesn’t mater if I follow this advice or not, a restart of the backup works and breaks again after some time.

Is there anything I can try to make this work?

My duplicati version is: Duplicati - 2.0.2.1_beta_2017-08-01

My system is this:

System properties

APIVersion : 1
PasswordPlaceholder : **********
ServerVersion : 2.0.2.1
ServerVersionName : - 2.0.2.1_beta_2017-08-01
ServerVersionType : Beta
BaseVersionName : 2.0.2.1_beta_2017-08-01
DefaultUpdateChannel : Beta
DefaultUsageReportLevel : Information
ServerTime : 2018-01-12T19:53:50.12442+01:00
OSType : Linux
DirectorySeparator : /
PathSeparator : :
CaseSensitiveFilesystem : true
MonoVersion : 4.6.2
MachineName : aswift
NewLine :
CLRVersion : 4.0.30319.42000
CLROSInfo : {"Platform":"Unix","ServicePack":"","Version":"4.10.0.42","VersionString":"Unix 4.10.0.42"}
ServerModules : []
UsingAlternateUpdateURLs : false
LogLevels : ["Profiling","Information","Warning","Error"]
SuppressDonationMessages : false
SpecialFolders : [{"ID":"%MY_DOCUMENTS%","Path":"/home/andre"},{"ID":"%MY_MUSIC%","Path":"/home/andre"},{"ID":"%MY_PICTURES%","Path":"/home/andre"},{"ID":"%DESKTOP%","Path":"/home/andre/Desktop"},{"ID":"%HOME%","Path":"/home/andre"}]
BrowserLocale : {"Code":"en-US","EnglishName":"English (United States)","DisplayName":"English (United States)"}
SupportedLocales : [{"Code":"de","EnglishName":"German","DisplayName":"Deutsch"},{"Code":"en","EnglishName":"English","DisplayName":"English"},{"Code":"es","EnglishName":"Spanish","DisplayName":"español"},{"Code":"fi","EnglishName":"Finnish","DisplayName":"suomi"},{"Code":"fr","EnglishName":"French","DisplayName":"français"},{"Code":"nl-NL","EnglishName":"Dutch (Netherlands)","DisplayName":"Nederlands (Nederland)"},{"Code":"pl","EnglishName":"Polish","DisplayName":"polski"},{"Code":"ru","EnglishName":"Russian","DisplayName":"русский"},{"Code":"zh-CN","EnglishName":"Chinese (Simplified)","DisplayName":"中文 (中国)"}]
BrowserLocaleSupported : true
backendgroups : {"std":{"ftp":null,"ssh":null,"webdav":null,"openstack":"OpenStack Object Storage / Swift","s3":"S3 Compatible","aftp":"FTP (Alternative)"},"local":{"file":null},"prop":{"s3":null,"azure":null,"googledrive":null,"onedrive":null,"cloudfiles":null,"gcs":null,"openstack":null,"hubic":null,"amzcd":null,"b2":null,"mega":null,"box":null,"od4b":null,"mssp":null,"dropbox":null,"jottacloud":null}}
GroupTypes : ["Local storage","Standard protocols","Proprietary","Others"]
Backend modules:

aftp

amzcd

azure

b2

box

cloudfiles

dropbox

ftp

file

googledrive

gcs

hubic

jottacloud

mega

onedrive

openstack

s3

ssh

od4b

mssp

tahoe

webdav
Compression modules:

zip

7z
Encryption modules:

aes

gpg

Server state properties

lastEventId : 144
lastDataUpdateId : 30
lastNotificationUpdateId : 13
estimatedPauseEnd : 0001-01-01T00:00:00
activeTask : {"Item1":8,"Item2":"4"}
programState : Running
lastErrorMessage :
connectionState : connected
xsfrerror : false
connectionAttemptTimer : 0
failedConnectionAttempts : 0
lastPgEvent : {"BackupID":"4","TaskID":8,"BackendAction":"List","BackendPath":null,"BackendFileSize":-1,"BackendFileProgress":0,"BackendSpeed":-1,"CurrentFilename":"/media/andre/andre_backup/bilder/2017/20170929_selin_cliff_hotel/20170929_131844_DSC01469.jpg","CurrentFilesize":5981040,"CurrentFileoffset":2048000,"Phase":"Backup_ProcessingFiles","OverallProgress":0,"ProcessedFileCount":2334,"ProcessedFileSize":12659743932,"TotalFileCount":64834,"TotalFileSize":278100596636,"StillCounting":false}
updaterState : Waiting
updatedVersion :
updateReady : false
updateDownloadProgress : 0
proposedSchedule : [{"Item1":"3","Item2":"2018-01-13T12:00:00Z"}]
schedulerQueueIds : []
pauseTimeRemain : 0

Thanks, André

2

Hi @andre_hollstein, welcome to the forum!

If you look at the main menu “Show log” page and select “Live” then “Profiling”, what commands do you see just before the failure point? (If you prefer, you could instead add --log-file=<path> and --log-level=profiling to your job then just view the file.)

My guess is it’s taking ~20 min. for Duplicati to scan your files and prep the first file for upload, at which point the failure occurs. If you edit your job on step 2 (Destination) and click “Test connection” do you get a success or failure message?

If so then you may be seeing a similar problem to the OAuth issues recently reported with Google Drive and OneDrive For Business, so this may be related. If not, and the break happens at “exactly” the some time each run then it may be an HTTP timeout issue.

3

Hi @JonMikelV, many thanks for your idea. It turns out that it works when I reduce the Upload volume size, it didn’t work with 100MB and 50MB, but with 30MB, the backup runs smoothly for hours and doesn’t break anymore.

I have checked on onedrive that files are arriving there, all looks good.

Would you expect this?

Cheers, André

4

If I recall correctly, HTTPS has a session data limit of 2GB. Might you be hitting this ?

5

Glad you found a workaround, though I’m pretty sure other users are using much larger than 30MB upload volume sizes, so I think it’s still something “odd” with your connection.

If you update to a newer version of Duplicati it supports the HTTP timeout parameter I earlier which would allow testing of large volume sizes.

Of course if you’re happy with what’s working then there no point in screwing around with it either. :slight_smile:

That’s a good point as well, though I want his internet if he’s getting 6GB / he. :smile:

I wonder if the --no-connection-reuse parameter would help get around that.

6

If I can help to discover a bug or unexpected behavior I’m glad to help. You mention a newer version of Duplicati, I was thinking I’m using the latest one: 2.0.2.1_beta_2017-08-01, is that not right?

How should I safely update?

7

Sorry for the delay - you are using the latest version in the beta channel. If you go to the main menu Settings page you should see a section called “Update channel” which probably has the Beta option selected. If you change that to Canary then you should start getting alerts about new versions there (I believe the latest is 2.0.2.14_canary_2017-12-22 in that channel).

Some things to note include:

  • a new beta version is supposed to come out “soon”
  • there is a change in how updates are handled between your version and later versions and SOME people have run into issues with the automatic update. When that happens a manual install of the new version generally seems to resolve it

Personally, I use the canary versions and other than the initial upgrade issue I mentioned above, haven’t had any issues.