Failed to process file...FileMissingException: File doesn't exist

lebrac · July 6, 2018, 7:09am

Hi guys,

For a couple of day, I get end backup messages saying that it failed to process some files (see below).

Is my backup corrupted? What should I do?

Any help would be appreciated

Thank you

**Jul 6, 2018 8:37 AM: Result**
DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 1
ExaminedFiles: 212388
OpenedFiles: 1
AddedFiles: 0
SizeOfModifiedFiles: 12292
SizeOfAddedFiles: 0
SizeOfExaminedFiles: 17069767621
SizeOfOpenedFiles: 12292
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
CompactResults:
    DeletedFileCount: 0
    DownloadedFileCount: 0
    UploadedFileCount: 0
    DeletedFileSize: 0
    DownloadedFileSize: 0
    UploadedFileSize: 0
    Dryrun: False
    MainOperation: Compact
    ParsedResult: Success
    EndTime: 06/07/2018 08:34:15 (1530858855)
    BeginTime: 06/07/2018 08:34:11 (1530858851)
    Duration: 00:00:03.7987070
    BackendStatistics:
        RemoteCalls: 13
        BytesUploaded: 18347367
        BytesDownloaded: 18360874
        FilesUploaded: 3
        FilesDownloaded: 2
        FilesDeleted: 1
        FoldersCreated: 0
        RetryAttempts: 4
        UnknownFileSize: 0
        UnknownFileCount: 0
        KnownFileCount: 462
        KnownFileSize: 11075971734
        LastBackupDate: 06/07/2018 08:14:48 (1530857688)
        BackupListCount: 8
        TotalQuotaSpace: 0
        FreeQuotaSpace: 0
        AssignedQuotaSpace: -1
        ReportedQuotaError: False
        ReportedQuotaWarning: False
        ParsedResult: Success
DeleteResults:
    DeletedSets: [
        Item1: 1
        Item2: 05/07/2018 19:30:00 (1530811800)
    ]
    Dryrun: False
    MainOperation: Delete
    ParsedResult: Success
    EndTime: 06/07/2018 08:34:15 (1530858855)
    BeginTime: 06/07/2018 08:34:05 (1530858845)
    Duration: 00:00:10.0620698
RepairResults: null
TestResults:
    MainOperation: Test
    Verifications: [
        Key: duplicati-20180706T061448Z.dlist.zip.aes
        Value: [],
        Key: duplicati-i6dd86e62101a4dd5916365533888b89c.dindex.zip.aes
        Value: [],
        Key: duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes
        Value: [
            Key: Error
            Value: Le fichier demandé n'existe pas      **that is to say: File doesn't exist**
        ]
    ]
    ParsedResult: Success
    EndTime: 06/07/2018 08:37:31 (1530859051)
    BeginTime: 06/07/2018 08:34:17 (1530858857)
    Duration: 00:03:14.5537304
ParsedResult: Error
EndTime: 06/07/2018 08:37:32 (1530859052)
BeginTime: 06/07/2018 08:14:44 (1530857684)
Duration: 00:22:47.4427603
Messages: [
    Deleting 1 remote fileset(s) ...,
    Deleted 1 remote fileset(s),
    Compacting not required
]
Warnings: []
Errors: [
    Failed to process file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes => Le fichier demandé n'existe pas
]

**Jul 6, 2018 8:37 AM: Message**
Failed to process file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.BackendManager.GetForTesting(String remotename, Int64 size, String hash)
   à Duplicati.Library.Main.Operation.TestHandler.DoRun(Int64 samples, LocalTestDatabase db, BackendManager backend)

**Jul 6, 2018 8:37 AM: Retry**
Operation Get with file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes attempt 5 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.BackendManager.GetForTesting(String remotename, Int64 size, String hash)
   à Duplicati.Library.Main.Operation.TestHandler.DoRun(Int64 samples, LocalTestDatabase db, BackendManager backend)

**Jul 6, 2018 8:37 AM: Retry**
Operation Get with file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes attempt 4 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

**Jul 6, 2018 8:37 AM: Retry**
Operation Get with file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes attempt 3 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

**Jul 6, 2018 8:37 AM: Retry**
Operation Get with file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes attempt 2 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

**Jul 6, 2018 8:37 AM: Retry**
Operation Get with file duplicati-b98d28eb06070420c9eb10382577375f9.dblock.zip.aes attempt 1 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

**Jul 6, 2018 8:34 AM: Message**
Compacting not required
Jul 6, 2018 8:34 AM: Message
Deleted 1 remote fileset(s)
Jul 6, 2018 8:34 AM: Message
Deleting 1 remote fileset(s) ...

JonMikelV · July 10, 2018, 6:11pm

I looks like the warnings may be coming as part of the post-backup verification step. If so, then the good news is your backups are likely running just fine.

Some questions I have include:

What version of Duplicati are you using and on what OS (Linux, MacOS, Windows…)?
Do the warnings happen with every run? (Do they go away if you set --backup-test-samples=0?)
When the warnings happen are they always for the same file? (Is it always a dblock.zip.aes file?)

lebrac · July 12, 2018, 1:52pm

Hi @JonMikelV,

What version of Duplicati are you using and on what OS (Linux, MacOS, Windows…)?

Duplicati - 2.0.3.3_beta_2018-04-02

Do the warnings happen with every run? (Do they go away if you set --backup-test-samples=0?)

No, I noticed these warnings a couple of days at the first run on the morning only and I didn’t see any new one till I post in this forum!?

When the warnings happen are they always for the same file? (Is it always a dblock.zip.aes file?)

I don’t remember but I think so…

What is the next step?

Thank you for your support

JonMikelV · July 15, 2018, 2:45pm

I forgot to ask what destination you are using, but it sounds like the issue is most likely that after completing a backup Duplicati is trying to test the files that were just uploaded and the destination isn’t ready to download them yet.

As test (not a fix) do the warnings stop if you add --backup-test-samples=0 to your job?

Note that this just tells Duplicati to NOT perform the post-backup test. Your backup itself should run as normal, but existing backups are no longer being tested to make sure they’re still valid.

lebrac · July 18, 2018, 7:27am

Hello @JonMikelV,

Sorry for the delay, I was out of my office these last two days…

The destination I use is hubiC, an online storage solution developed by the OVH group, a French company. This solution offers a large storage space and is cheap, but it is not renowned for its upload speed which is very slow… as well as my internet connection…

Surprisingly, I have no more error message. Has hubiC services be improved? Or were the servers overloaded at that time? I’m going to wait for the next error messages and then I’ll use --backup-test-samples = 0 for testing purposes.

In the meantime, is there a way to make sure the backup is OK?

Thank you for your help.

dapobe · July 18, 2018, 8:29am

I’m using Hubic and have fails on every backup for some days.

Duplicati gives several error 503 ( server not ready). Other times Duplicate find files in remote storage that are not in the local storage. And sometimes fails trying to delete some files on remote site.

I have 2 backups to Hubic, one is a server and the other is my laptop. Both of them fails.

Hubic don’t accept new users. OVH will maintain users, but will not accept new ones. And they will not improve their infraestructure.

So, I think that Hubic is not a good place for your backups…

lebrac · July 18, 2018, 9:23am

Hi @dapobe,

You are right, hubiC will not improve their solution,because they stopped to sell it! And that’s a shame.

hubiC is a French storage with servers in France under French law which protects personal data, main reason why I subscribed.

I tried iKoula, another French storage which has little bit better upload speed but didn’t convince me and is less cheap.

I will therefore continue to use hubiC waiting for a better solution …

dapobe · July 18, 2018, 9:34am

Maybe Jottacloud, a norwegian provider. But I’ve read that restores are slow ( Duplicati use 1 trhead and Jottacloud own software uses 6 threads to download data).

Another solution is Amazon S3 using an european site ( Ireland, Frankfurt, etc…). S3 is, by the way, the cheapest solution.