Hi,
When I want to run a test restore via the GUI (Expand backup job/Advanced/Commandline) I do a restore and add the option --no-local-blocks but when restoring I get “Restore completed without errors but no files were restored”.
Full options given:
I should add that the “/(path)/(to)/(folder)” argument given above points to a folder and not a file. But I tried adding / after it and that didn’t work.
Log output is this:
Restore started at 01/10/2022 18:57:12
The operation Restore has started
Checking remote backup ...
Backend event: List - Started: ()
Listing remote folder ...
Backend event: List - Completed: (2.13 KB)
Searching backup 0 (01/10/2022 15:38:00) ...
Searching backup 1 (11/28/2021 03:27:45) ...
Searching backup 2 (09/19/2021 03:16:01) ...
Searching backup 3 (07/18/2021 03:16:11) ...
Searching backup 4 (05/14/2021 13:08:07) ...
Mapping restore path prefix to "" to "/src/homes/(SNIP)/"
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
0 files need to be restored (0 bytes)
Verifying restored files ...
Restore completed without errors but no files were restored
Email sent successfully using server: (SNIP)
Restored 0 (0 bytes) files to /src/homes/(SNIP)
Duration of restore: 00:00:02
Return code: 2
When I try to do this with the Duplicati commandline tool I have the same results.
I am running Duplicati - 2.0.6.3_beta_2021-06-17 in Docker on a Synology NAS.
This could very well be a small thing that I am doing wrong, question is, what? Google search didn’t help me further.
I can reproduce this problem but it doesn’t seem to have anything to do with the --no-local-blocks option.
I’m not really sure what’s going on yet but I can’t get the web UI command line restore to work, but a true command line restore does (with the same options).
Ok figured it out. Don’t put quotes around the files to restore when using the web UI command line.
Quotes are called for when you use the true command line (but I think they are probably only required if a space is present).
In the web UI command line, no quotes are ever used even if there’s a space present. The web UI uses hard returns as breaks between command line parameters. (You put each parameter on a separate line.)
Specific files in the backup set, but I specify a directory (hoping that it will restore everything that’s in it).
Syntax is /directory1/directory2/directory3/directory4/directory5 - with or without double quotes around it.
Interesting thing with Find is that if I specify the directories, nothing is found. If I specify files in the directory, find files them.
However, this does not solve the Restore issue, I still get following output:
Restore started at 02/26/2022 13:43:17
The operation Restore has started
Checking remote backup ...
Backend event: List - Started: ()
Listing remote folder ...
Backend event: List - Completed: (1.82 KB)
Searching backup 0 (09/19/2021 01:00:01) ...
Searching backup 1 (08/15/2021 01:00:19) ...
Searching backup 2 (08/08/2021 20:05:34) ...
Searching backup 3 (08/08/2021 11:09:48) ...
Searching backup 4 (07/11/2021 10:33:29) ...
Mapping restore path prefix to "" to "/src/homes/(SNIP)/"
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
Verifying restored files ...
Restore completed without errors but no files were restored
Email sent successfully using server: (SNIP)
Restored 0 (0 bytes) files to /src/homes/(SNIP)
Duration of restore: 00:00:02
Return code: 2
So, still no restore. Also, nothing happens in the destination folder /src/homes/(SNIP)
I don’t think that works, or at least it doesn’t with a find (as you saw). Try it with wildcard /* at end.
If you mean you have a working find, but restore has no restore list, odd. Works here on Windows Command Prompt. Read well. Duplicati won’t restore files if the destination already has the right files.
Why would you move the original files? You don’t want to clobber them with an old backup, so choose
and to ensure nothing comes from the original files, but only from the remote files, use no-local-blocks.
It’s on Options screen 5 in Advanced options, and should apply to standard GUI Restore automatically.
I know…and that behaviour is what I want to avoid by using the --no-local-blocks option.
I would move the original files because of Duplicati’s standard behavior that no restore happens if the original files are present and ok. In itself that behaviour makes sense, but not in case of a test restore - which is what I want to do here.
When I run the restore via the GUI and set the no-local-blocks option via the GUI in the general settings, it works as expected.
I can live with that, but still the question is, why not via a command line (even if via the command line GUI)?
The “no-local-blocks” option is something nice for a test setting, but not for real-life use (at least not in my case).
Therefore I think it would better suit my needs if I can put this option “just for now” and not as a general setting, which you could potentially forget to disable.
And the GUI-based commandline seems ideal for that. Except if you have a better idea…
But as stated, if this is not possible, I can live with that. Big thanks for your help!
These are not the same optimizations. See the --no-local-blocks talk of “source files”.
This refers to the original location which was backed up, and not the --restore-path folder.
I don’t think there’s an option to force Duplicati to re-restore files that are already just fine.
That’s why the default is off – unless you’re testing – but I agree one could forget to flip it.
It should work on a GUI or true command line, but you still need to clear the restore folder.
I think we are missing eachother at some point.
When restoring, I choose a restore folder where the files are not present (so, not the location where the files are stored originally). By using the --no-local-block option I hope I don’t have to manually move the original files, in the original location, in order to “force” Duplicati to restore from the backup (cloud) location instead of just copying it from the original location.
So, more concretely:
Original location: MyOriginalFiles/file1.dat
Restore location: MyRestoredFiles/file.dat
Backup location: MyCloudFiles/file.dat
In this scenario, MyRestoredFiles/file.dat is not present, but MyOriginalFiles/file1.dat is.
My aim is to do a test restore (ensuring the quality and reliability of MyCloudFiles/file.dat) and do so without having to manually move MyOriginalFiles/file1.dat to some temporary location. For this I want to use --no-local-blocks.
It is by doing that via the command line that I get the “Restore completed without errors but no files were restored” message.
As stated, I don’t care putting the (global) GUI parameter for no-local-blocks (I’ll just put a warning for myself not to forget to unset the parameter) but I guess this is still an issue.
It’s never present. Duplicati doesn’t put file copies in the destination. The file format is hugely different. Regardless, let’s get back to original and restore locations.
Recall there have been separate usage issues. What happens if you don’t use --no-local-blocks?
The only way I can cause that error message is if I incorrectly specify files that I want to have restored.
Note that if you’re using a *, you need to put the specifier in quotes, otherwise your shell will expand it.
EDIT:
$ pwd
/tmp/MyRestoredFiles
$ rm file1.dat
$ mono /usr/lib/duplicati/Duplicati.CommandLine.exe restore file:///tmp/MyDestinationFiles/ "/tmp/MyOriginalFiles/*" --dbpath=/tmp/LGIJWQCDDQ.sqlite --no-encryption=true --restore-path=/tmp/MyRestoredFiles --console-log-level=information
Restore started at 3/1/2022 4:43:42 PM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started: ()
Listing remote folder ...
Backend event: List - Completed: (3 bytes)
Searching backup 0 (3/1/2022 9:31:24 PM) ...
Checking existing target files ...
1 files need to be restored (5 bytes)
Scanning local files for needed data ...
1 remote files are required to restore
Backend event: Get - Started: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
Downloading file (574 bytes) ...
Backend event: Get - Completed: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 1 (5 bytes) files to /tmp/MyRestoredFiles
Duration of restore: 00:00:03
***********************************************
Did we help save your files? If so, please support Duplicati with a donation. We suggest 10€ for private use and 100€ for commercial use.
https://www.duplicati.com/donate/
***********************************************
$ rm file1.dat
$ mono /usr/lib/duplicati/Duplicati.CommandLine.exe restore file:///tmp/MyDestinationFiles/ "/tmp/MyOriginalFiles/*" --dbpath=/tmp/LGIJWQCDDQ.sqlite --no-encryption=true --restore-path=/tmp/MyRestoredFiles --console-log-level=information --no-local-blocks
Restore started at 3/1/2022 4:44:06 PM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started: ()
Listing remote folder ...
Backend event: List - Completed: (3 bytes)
Searching backup 0 (3/1/2022 9:31:24 PM) ...
Checking existing target files ...
1 files need to be restored (5 bytes)
1 remote files are required to restore
Backend event: Get - Started: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
Downloading file (574 bytes) ...
Backend event: Get - Completed: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 1 (5 bytes) files to /tmp/MyRestoredFiles
Duration of restore: 00:00:03
***********************************************
Did we help save your files? If so, please support Duplicati with a donation. We suggest 10€ for private use and 100€ for commercial use.
https://www.duplicati.com/donate/
***********************************************
$
Notice how adding --no-local-blocks stops Scanning local files for needed data ... step.
That’s what you should look at to see whether the option works. I think metadata is always downloads.
Note that with or without --no-local-blocks, I always get that file. Maybe you are asking differently?
--no-local-blocks with --restore-path is working for me through the command line as expected (I think).
All the files are correctly restored to the specified path, the Scanning local files for needed data … line is omitted from the command’s output and it takes a bit longer than when restoring without using the --no-local-blocks option (I’ve tested this with just 150MB data). I guess this is because it downloads all the data instead of reusing the data in the original source path?
What I don’t understand though is why, when not using --no-local-blocks and nothing has changed to the original source files (they are exactly the same as the remote backup), it still downloads some dblocks from the remote (although fewer than when using --no-local-blocks).
Anyway I think your problem is the "/(path)/(to)/(folder)" part. When specifying folders to backup or restore through the command line you need to add two trailing slashes otherwise it isn’t going to work: "/(path)/(to)/(folder)//".
This doesn’t apply to the --restore-path option for some reason…
EDIT: The two trailing slashes rule applies for Windows at least, I haven’t used Duplicati on Linux yet… "C:\(path)\(to)\(folder)\\".
I’m a new user of Duplicati so I’m still struggling to understad how it works. I can’t find much information about metadata in the documentation, but according to this article I suppose they are stored in filelist.json inside the dlist file?
If yes then their size should be tiny, but I see from the command output that actual dblocks are downloaded (data of approximately 50MB).
PS. I can’t yet read code because it’s filled with ambiguous terms for me. For example I see everywhere mentions of “block” but I think this word is used interchangeably for either dblocks/volumes or, well… “blocks” aka chunks of 100KB. Which adds to my confusion.
Most people don’t care about the internals, but since we’re here and you ask
At article bottom, it says that “Metadata is treated like a normal block of data”.
Duplicati has block’s hash information, and uses it to know if source changed.
If source hash is as it was at backup time, the source block is used to restore.
I “think” the comment is saying it “could” do that for metadata, but doesn’t yet.
On Windows, it’s typically about 100 bytes, compared to 100 KB typical data.
Duplicati keeps many blocks in a dblock file packed until full (default 50 MB).
On the Options screen 5, this size is configurable by Remote volume size.
