Error While running: The requested file does not exist

lebrac · October 11, 2018, 9:15am

After a long, trouble-free period, all backups ended in error in the last few days.

I tried several options such as repairing the database, checking files, repairing again, … Unfortunately, this has not changed anything for backups that still crash.

In desperation I performed the recreation (deletion and repair) of the database. Alas, it also crashes with the message “The requested file does not exist” (see history below).

I am blocked now and I don’t know what to do to restore the situation and make the backups work again (I have about 10 GB of stored data and I would like to avoid having to recreate everything!).

Thank you in advance for any help…

11 oct. 2018 09:20: Message
---------------------------
Probing 4 candidate blocklist volumes

11 oct. 2018 09:08: Message
---------------------------
Processing required 16 blocklist volumes
11 oct. 2018 08:57: Message
Failed to process index file: duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:57: Retry
---------------------------
Operation Get with file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes attempt 5 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:57: Retry
---------------------------
Operation Get with file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes attempt 4 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

11 oct. 2018 08:57: Retry
---------------------------
Operation Get with file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes attempt 3 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:57: Retry
---------------------------
Operation Get with file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes attempt 2 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:57: Retry
---------------------------
Operation Get with file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes attempt 1 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:51: Message
---------------------------
Failed to process index file: duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:51: Retry
---------------------------
Operation Get with file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes attempt 5 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:51: Retry
---------------------------
Operation Get with file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes attempt 4 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:51: Retry
---------------------------
Operation Get with file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes attempt 3 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()

11 oct. 2018 08:51: Retry
---------------------------
Operation Get with file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes attempt 2 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:51: Retry
---------------------------
Operation Get with file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes attempt 1 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:43: Message
---------------------------
Failed to process index file: duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:43: Retry
---------------------------
Operation Get with file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes attempt 5 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:43: Retry
---------------------------
Operation Get with file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes attempt 4 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:43: Retry
---------------------------
Operation Get with file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes attempt 3 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:43: Retry
---------------------------
Operation Get with file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes attempt 2 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:43: Retry
---------------------------
Operation Get with file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes attempt 1 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:39: Message
---------------------------
Failed to process index file: duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:39: Retry
---------------------------
Operation Get with file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes attempt 5 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()
   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)
   
11 oct. 2018 08:39: Retry
---------------------------
Operation Get with file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes attempt 4 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:39: Retry
---------------------------
Operation Get with file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes attempt 3 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:39: Retry
---------------------------
Operation Get with file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes attempt 2 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:39: Retry
---------------------------
Operation Get with file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes attempt 1 of 5 failed with message: Le fichier demandé n'existe pas
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas (_that is The requested file does not exist_)
   à Duplicati.Library.Backend.OpenStack.OpenStackStorage.Get(String remotename, Stream stream)
   à Duplicati.Library.Main.BackendManager.coreDoGetPiping(FileEntryItem item, IEncryption useDecrypter, Int64& retDownloadSize, String& retHashcode)
   à Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
   à Duplicati.Library.Main.BackendManager.ThreadRun()
   
11 oct. 2018 08:37: Message
---------------------------
Filelists restored, downloading 383 index files

11 oct. 2018 08:32: Message
---------------------------
Rebuild database started, downloading 10 filelists

lebrac · October 11, 2018, 9:32am

After the last execution of the “delete and recreate database” command, which ended in error, I executed the “check files” command which ended with the following errors:

ParsedResult: Error
EndTime: 11/10/2018 11:07:21 (1539248841)
BeginTime: 11/10/2018 10:55:49 (1539248149)
Duration: 00:11:32.3273824
Messages: []
Warnings: []
Errors: [
    Failed to process file duplicati-b42998acb9b5e4295902b69279fc91d32.dblock.zip.aes => The requested file does not exist,
    Failed to process file duplicati-i0070fa5983b640e0bbaa3943807ff2df.dindex.zip.aes => The requested file does not exist,
    Failed to process file duplicati-i2a73d836379e4f8fb42064d2d9012a07.dindex.zip.aes => The requested file does not exist,
    Failed to process file duplicati-i71f3af27108b49eaac943c406a98ba2e.dindex.zip.aes => Unexpected character encountered while parsing value: <. Path '', line 0, position 0.,
    Failed to process file duplicati-i8cbc9363ff3b48b2a2f363445d65e389.dindex.zip.aes => The requested file does not exist
]

How to identify these files? Can they be removed without problem?

Thanks for your help…

lebrac · October 18, 2018, 1:16pm

Up!

Is there someone (Duplicati Team?) who can help?

ts678 · October 18, 2018, 3:01pm

I’m not “Duplicati Team” (this is a user forum) but I’ll try. What Duplicati and OS is this?

Do you have logs or memories about the original error? Was it similar to the deletion/repair errors shown?

Does “check files” refer to the GUI Verify files item which is a small changing sampling, or something else?

Can you look at the destination and see whether the named files (do they vary on repeated test?) exist, or are you saying (because you’re asking about removal) you see them but Duplicati is seeing differently?

The AFFECTED command can show what source files would be affected if destination files were removed. If you normally run from the web UI, the easiest thing to do is to run the job Commandline option and adjust format for new command. This will automatically bring in other needed options such as destination, encryption, etc. Disaster Recovery has information on how to run this, and can help with severe issues.

Visual inspection of backend files might be helpful. I think you would have a dated dlist file for each backup version, and an equal number of dblock and dindex files. Their names are random, but a linkage is inside.

You can also get a look at your destination through Duplicati without the database, by using the MENU → Restore → Direct restore from backup files GUI option from the UI you normally work in. Does it look OK?

lebrac · October 29, 2018, 11:04am

Hello @ts678

I know this is an user forum, but I was hoping that Duplicati representatives would also consult it…

And I appreciate your help very much

Duplicati version is 2.0.3.3_beta_2018-04-02 and OS: Windows 10 Professional version 1709 (16299.665)

No, because I I tried many options before attempting to recreate the database and I didn’t keep the logs.

Following some of your advice (I’m not sure to have understood others, sorry…), I have used the AFFECTED command to check the “missing” files and the result is the same for all:

Database file does not exist: C:\WINDOWS\system32\config\systemprofile\AppData\Local\Duplicati\OFVRTJZUVU.sqlite
Return code: 100

Actually, looking at the C:\WINDOWS\system32\config\systemprofile\AppData\Local\Duplicati\ folder I do’nt find any OFVRTJZUVU.sqlite like noted in the error message but:

a KILBDQZSOO.sqlite and a KILBDQZSOO.backup files
a RVXREKXJAU.sqlite file
a dbconfig.json and a Duplicati.server.sqlite files
a control_dir_v2 folder

Does this information help?

I tried this option and I have successfully restored one file from the last backup. But the operation ended with 5 warnings (see restore log attached to this post). These 5 warnings correspond to the always same 5 missing files …

Is there a safe way to get rid of these ghost files?

Thank you

duplicati-restore-log.zip (1.9 KB)

ts678 · October 29, 2018, 8:41pm

I think this repair effort is in a rather hard spot. Ordinarily commands like “affected” and “list-broken-files” can help, but they seem to require a database (and –no-local-db, true to its documentation, wasn’t helping them). The destination and database should track each other, and can repair each other (sometimes) but yours are both having problems. The other randomly-named database files would probably only be interesting if they’re recent enough to help the current situation. If you have other backup jobs, also check their Database names.

One avenue that sometimes works if a backup makes a bad version is to delete that version (generally zero). Having no database to do that on, the equivalent would be to see if The REPAIR command can recreate any versions before the last unfortunate one. The documentation doesn’t say what options it takes, but test finds that it takes –version which is just what we want. So to skip the latest, avoid 0 and set a big range like 1-999. Depending on your preference, you can export the job as a command line and run it from Command Prompt after adjusting to add --version and remove the source file list, or do the same thing in job Commandline GUI. Using either method, you should start without a database. Save ones that might have value. Delete hard fails.

Possibly this will let you recreate enough of the database to not lose all the versions, and the next step would be to stop the newer destination files from surprising the backup. I think that’s easier than the current rebuild.

If you don’t want to go right to the database mostly-recreate, you can also use the direct restore to see if the version before the latest loses the complaint about the missing index files. A smooth restore is a good sign…

Your original post sounded like the main worry was recreating everything, not loss of old versions, or newest, however you can speak to that further if you like. If just guarding against disaster, sometimes a restart is fine.

lebrac · October 31, 2018, 10:09am

Yes, you’re right !

That’s why, after spending too many time trying to recover this backup, I decided to give up …

In desperation, I decided to recreate the backup from scratch - 17GB to save - and try one last time to use Duplicati. If it crashes again, I will have to find another solution; too bad for Duplicati…

It is a pity that the authors of Duplicati are so little concerned by the support of their software, relying only on the good will of users to find solutions to problems encountered …

In the meantime, thank you @ts678 for your help