Odd errors using box.com

Hi all,

Was a crashplan and previous duplicati 1.3.4 user. After the demise of crashplan did the dive into the ‘beta’ version. So far still testing out but really appreciate the new cloud services supported.

Something weird though - seems other cloud providers work ok but for Box.com - I end up getting strange error messages (“Got 0 warnings”).

Screenshot and some logs I could see provided. Anybody have insights why this is happening?

DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 1
ExaminedFiles: 33
OpenedFiles: 1
AddedFiles: 0
SizeOfModifiedFiles: 5896192
SizeOfAddedFiles: 0
SizeOfExaminedFiles: 9905308
SizeOfOpenedFiles: 5896192
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
CompactResults:
    DeletedFileCount: 0
    DownloadedFileCount: 0
    UploadedFileCount: 0
    DeletedFileSize: 0
    DownloadedFileSize: 0
    UploadedFileSize: 0
    Dryrun: False
    MainOperation: Compact
    ParsedResult: Success
    EndTime: 3/09/2017 8:19:51 AM
    BeginTime: 3/09/2017 8:19:51 AM
    Duration: 00:00:00.0030024
    BackendStatistics:
        RemoteCalls: 20
        BytesUploaded: 5237735
        BytesDownloaded: 0
        FilesUploaded: 3
        FilesDownloaded: 0
        FilesDeleted: 0
        FoldersCreated: 0
        RetryAttempts: 12
        UnknownFileSize: 0
        UnknownFileCount: 0
        KnownFileCount: 6
        KnownFileSize: 13619854
        LastBackupDate: 3/09/2017 4:19:37 PM
        BackupListCount: 2
        TotalQuotaSpace: 0
        FreeQuotaSpace: 0
        AssignedQuotaSpace: -1
        ParsedResult: Success
DeleteResults: null
RepairResults: null
TestResults:
    MainOperation: Test
    Verifications: [
        Key: duplicati-20170903T075702Z.dlist.zip.aes
        Value: [
            Key: Error
            Value: The requested file does not exist
        ],
        Key: duplicati-ife843b266bc34aa4a1a052869b4d6b69.dindex.zip.aes
        Value: [
            Key: Error
            Value: The requested file does not exist
        ],
        Key: duplicati-b477f2241fe974921824b4955139c59c7.dblock.zip.aes
        Value: [
            Key: Error
            Value: The requested file does not exist
        ]
    ]
    ParsedResult: Success
    EndTime: 3/09/2017 8:22:08 AM
    BeginTime: 3/09/2017 8:19:52 AM
    Duration: 00:02:15.9896801
ParsedResult: Error
EndTime: 3/09/2017 8:22:08 AM
BeginTime: 3/09/2017 8:19:37 AM
Duration: 00:02:31.7372402
Messages: [
    Compacting not required
]
Warnings: []
Errors: [
    Failed to process file duplicati-20170903T075702Z.dlist.zip.aes => The requested file does not exist,
    Failed to process file duplicati-ife843b266bc34aa4a1a052869b4d6b69.dindex.zip.aes => The requested file does not exist,
    Failed to process file duplicati-b477f2241fe974921824b4955139c59c7.dblock.zip.aes => The requested file does not exist
]

I am getting the same error messages and will investigate why it happens.

Any luck finding out about that “The requested file does not exist” error (and resulting “Got 0 warning(s)” message) associated with Box.com?

I ask because I’m getting the same thing on a test backup, even though I confirmed the file actually exists.

No, I did not get to test it. Too much stuff happening :slight_smile:

Just to be sure and not to waste time on more investigation: box.com can still not be used as a backup target without getting this errors?

I get the same errors. Files exists. So I just ignore it and will not use box.

Any news here on this error?

Not from my end. My experience with it was just with a test backup which has since gone away (forgot to save it before wiping a machine).

Knowing more about Duplicati now than I did back then my guess is that there’s either a bug in how box.com reports (via their API) files or a race condition such that a file recently uploaded to them isn’t reported (via their API) as existing if queried too soon after creation.

Since mine was very small test backup I believe I had a single archive (dblock) and “what’s-been-backed-up” (dlist) file set created. This meant that the automatic “test a single random archive file” step Duplicati does after a backup had to choose the only file that existed (and in this case was recently uploaded).

Do you know how many dblock files you have at your destination?

If I (or you) were to test this theory, some things to try out would include:

  • is the destination create date on the “requested file does not exist” error very recent?
  • do you have less than 5 or so dblock files?
  • does the error only happen sometimes (and does the error frequency drop as the number of dblock files increases)?
  • does enabling --no-backend-verification PURELY AS A TEST stop the errors from happening?

If any (or most) of the above are “yes” then it’s most likely a race condition (Duplicati is asking for the file list faster than the destination is able to provide an up-to-date one).

@kenkendk, what do you think of retiring --no-backend-verification and replacing it with something like --backend-validation=before|after|none?

Considerations include:

  • Con: with the before setting on initial backups (esp. of small data sets) this potentially exposes a user to a bad backup not being detected until the next job run
  • Pro: should fully remove the related “file does not exist” scenario when caused by race conditions

Quick data point - I have --no-backend-verification set for other reasons (make backups faster coz the check increases the turn around time by a lot)

I used to have the box.com error but have noticed it hasnt come up for a while. So perhaps the theory is right (but take with a grain of salt since all anecdotal and not really scientifically tested)

Thanks for the follow-up!

I had also problems with box.com. Every backup reported an error.
Then I changed from “box.com” to “webdav” and now it works without this errors?!?

Maybe you will try it with webdav?

Settings: Use SSL, dav.box.com, 443, /dav/duplicati [after /dav/ you can choose your folder]

Disadvantage is that you have to use user+password instead of the apikey.

Thanks for the suggestion @thommyX! :+1:

I could have sworn I saw another post where somebody else reported using WebDAV worked better for them than the box.com API as well, but I can’t seem to find it now. :frowning:

That would have been me. And yes webdav works better and faster as well if I recall correctly

1 Like

Quick follow up:

I found and fixed the error in the box.com backend, and I also found and fixed the “0 warning(s)” dialog. The fixes will be part of the next canary build.

2 Likes

Is it possible that box.com is no longer usabel in Duplicati? Can anyone post if he/she has the same problems or if it works?

Have a complete empty box.com free account.
Setup a new Backup in Duplicati - tried both webdav (SSL, dav.box.com:443 and path /dav/duplicati) and box.com with API key

I use Duplicati beta from Nov. Error is the same in both setups

Backup started at 20.07.2019 10:32:15
Checking remote backup ...
  Listing remote folder ...
  Listing remote folder ...
  Listing remote folder ...
  Listing remote folder ...
  Listing remote folder ...
Fatal error => The underlying connection was closed: An unexpected error occurred on a send.

System.Net.WebException: The underlying connection was closed: An unexpected error occurred     on a send. ---> System.IO.IOException: Unable to read data from the
transport connection: An existing connection was forcibly closed by the remote host. --->     System.Net.Sockets.SocketException: An existing connection was forcibl
y closed by the remote host
   at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
   at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
   --- End of inner exception stack trace ---
   at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
   at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
   --- End of inner exception stack trace ---
   at Duplicati.Library.Main.BackendManager.List()
   at Duplicati.Library.Main.Operation.FilelistProcessor.RemoteListAnalysis(BackendManager     backend, Options options, LocalDatabase database, IBackendWriter log,
 String protectedfile)
   at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend,     Options options, LocalDatabase database, IBackendWriter log, S
tring protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend,     String protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)
   at Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)
   at Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1     method)
   at Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)
   at Duplicati.CommandLine.Commands.Backup(TextWriter outwriter, Action`1 setup, List`1 args,     Dictionary`2 options, IFilter filter)
   at Duplicati.CommandLine.Program.ParseCommandLine(TextWriter outwriter, Action`1 setup,     Boolean& verboseErrors, String[] args)
   at Duplicati.CommandLine.Program.RunCommandLine(TextWriter outwriter, TextWriter errwriter,     Action`1 setup, String[] args)
Update "2.0.4.23_beta_2019-07-14" detected

Box Com webdav duplicati 1.3.4 remote host closed connection #3817 reports duplicati 2 works.

What OS are you using? If you’re on an old one such as Windows 7, this might be TLS versions:

Updating Box from the Transport Layer Security (TLS) 1.0 Encryption Protocol FAQ

On Windows, see if Internet Explorer can get to https://update.box.com and https://dav.box.com.

Internet options --> Advanced has some TLS checkboxes you can test to see what your OS has.

–allowed-ssl-options is Duplicati’s control. If your OS can do newer TLS, this is how to ask for it.

Update - WebDAV Support is also a warning to any Box WebDAV users. Service may end soon:

We originally announced the end of our support for WebDAV effective on Jan 31, 2019. We are extending support for WebDAV beyond our originally-announced date.

1 Like

Setting the allowed-ssl-options to Version 1.1 did the job! Thank you!