--no-local-blocks not working - still scans for local blocks

Hi,
When I want to run a test restore via the GUI (Expand backup job/Advanced/Commandline) I do a restore and add the option --no-local-blocks but when restoring I get “Restore completed without errors but no files were restored”.
Full options given:

"/(path)/(to)/(folder)"
--restore-path=/src/homes/(SNIP)
--no-local-blocks=true
--console-log-level=Verbose
--full-result
--send-mail-to=(SNIP)
--send-mail-url=(SNIP)
--send-mail-username=(SNIP)
--send-mail-password=(SNIP)
--send-mail-from=(SNIP)
--auto-vacuum=true
--backup-test-percentage=5
--send-mail-any-operation=true
--send-mail-level=all
--backup-name=(SNIP)
--dbpath=(SNIP)
--encryption-module=aes
--compression-module=zip
--dblock-size=500MB
--passphrase=(SNIP)
--keep-versions=5
--blocksize=100MB
--log-file=(SNIP)
--log-file-log-level=Retry
--disable-module=console-password-input

I should add that the “/(path)/(to)/(folder)” argument given above points to a folder and not a file. But I tried adding / after it and that didn’t work.

Log output is this:

Restore started at 01/10/2022 18:57:12
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (2.13 KB)
Searching backup 0 (01/10/2022 15:38:00) ...
Searching backup 1 (11/28/2021 03:27:45) ...
Searching backup 2 (09/19/2021 03:16:01) ...
Searching backup 3 (07/18/2021 03:16:11) ...
Searching backup 4 (05/14/2021 13:08:07) ...
Mapping restore path prefix to "" to "/src/homes/(SNIP)/"
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restore completed without errors but no files were restored
Email sent successfully using server: (SNIP)
Restored 0 (0 bytes) files to /src/homes/(SNIP)
Duration of restore: 00:00:02
Return code: 2

When I try to do this with the Duplicati commandline tool I have the same results.

I am running Duplicati - 2.0.6.3_beta_2021-06-17 in Docker on a Synology NAS.

This could very well be a small thing that I am doing wrong, question is, what? Google search didn’t help me further.

Any extra help is much appreciated!

I can reproduce this problem but it doesn’t seem to have anything to do with the --no-local-blocks option.

I’m not really sure what’s going on yet but I can’t get the web UI command line restore to work, but a true command line restore does (with the same options).

Ok figured it out. Don’t put quotes around the files to restore when using the web UI command line.

Quotes are called for when you use the true command line (but I think they are probably only required if a space is present).

In the web UI command line, no quotes are ever used even if there’s a space present. The web UI uses hard returns as breaks between command line parameters. (You put each parameter on a separate line.)

Thanks, but that doesn’t seem to solve it. I tried it without any quotes but still no avail:

Restore started at 02/26/2022 11:21:22
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (2.14 KB)
Searching backup 0 (01/16/2022 03:18:16) ...
Searching backup 1 (01/10/2022 15:38:00) ...
Searching backup 2 (11/28/2021 03:27:45) ...
Searching backup 3 (09/19/2021 03:16:01) ...
Searching backup 4 (07/18/2021 03:16:11) ...
Mapping restore path prefix to "" to "/src/homes/(SNIP)/"
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restore completed without errors but no files were restored
Email sent successfully using server: (SNIP)
Restored 0 (0 bytes) files to /src/homes/(SNIP)
Duration of restore: 00:00:04
Return code: 2

Same when I try to do it in the command line (in this case with qoutes as the path contains spaces).

Any other ideas?

Are you trying to restore everything, or specific files? If the latter, what sort of syntax did you use?

You could test changing restore to find to see what it finds. If nothing, that explains lack of restore.

Restoring files from a backup is the simpler way. Specifying files for Commandline is more difficult.

Specific files in the backup set, but I specify a directory (hoping that it will restore everything that’s in it).
Syntax is /directory1/directory2/directory3/directory4/directory5 - with or without double quotes around it.

Interesting thing with Find is that if I specify the directories, nothing is found. If I specify files in the directory, find files them.
However, this does not solve the Restore issue, I still get following output:

Restore started at 02/26/2022 13:43:17
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (1.82 KB)
Searching backup 0 (09/19/2021 01:00:01) ...
Searching backup 1 (08/15/2021 01:00:19) ...
Searching backup 2 (08/08/2021 20:05:34) ...
Searching backup 3 (08/08/2021 11:09:48) ...
Searching backup 4 (07/11/2021 10:33:29) ...
Mapping restore path prefix to "" to "/src/homes/(SNIP)/"
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
Verifying restored files ...
Restore completed without errors but no files were restored
Email sent successfully using server: (SNIP)
Restored 0 (0 bytes) files to /src/homes/(SNIP)
Duration of restore: 00:00:02
Return code: 2

So, still no restore. Also, nothing happens in the destination folder /src/homes/(SNIP)

I can see that, but the aim is to run a test restore in which I want to test the reliability of the backups (in case I get some errors during backup verification, see Backup test errors - The file duplicati-(...).dblock.zip.aes was downloaded and had size X but the size was expected to be Y - #19 by Bart80.2). And if I can do that without manually having to move the original files around, it would be so much easier. And I did not yet find a way to do this in the restore GUI…

I don’t think that works, or at least it doesn’t with a find (as you saw). Try it with wildcard /* at end.

If you mean you have a working find, but restore has no restore list, odd. Works here on Windows Command Prompt. Read well. Duplicati won’t restore files if the destination already has the right files.

Why would you move the original files? You don’t want to clobber them with an old backup, so choose

image

and to ensure nothing comes from the original files, but only from the remote files, use no-local-blocks.
It’s on Options screen 5 in Advanced options, and should apply to standard GUI Restore automatically.

Correct - that works.
Parameters:

/dir1/dir2/dir3/dir4/dir5/*
--console-log-level=Verbose
--full-result

Log:

The operation List has started
Email sent successfully using server: (SNIP)
Listing contents 0 (01/16/2022 03:18:16):
/dir1/dir2/dir3/dir4/dir5/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file101.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file101.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file102.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file102.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file103.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file103.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file104.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file104.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file105.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file105.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file106.dat/ 
/dir1/dir2/dir3/dir4/dir5/@eaDir/fn - file106.dat/metadata (481 bytes)
/dir1/dir2/dir3/dir4/dir5/fn - file101.dat (1.15 GB)
/dir1/dir2/dir3/dir4/dir5/fn - file102.dat (1.07 GB)
/dir1/dir2/dir3/dir4/dir5/fn - file103.dat (1.26 GB)
/dir1/dir2/dir3/dir4/dir5/fn - file104.dat (1.09 GB)
/dir1/dir2/dir3/dir4/dir5/fn - file105.dat (1.08 GB)
/dir1/dir2/dir3/dir4/dir5/fn - file106.dat (1.18 GB)
Return code: 0

I know…and that behaviour is what I want to avoid by using the --no-local-blocks option.

I would move the original files because of Duplicati’s standard behavior that no restore happens if the original files are present and ok. In itself that behaviour makes sense, but not in case of a test restore - which is what I want to do here.

When I run the restore via the GUI and set the no-local-blocks option via the GUI in the general settings, it works as expected.

I can live with that, but still the question is, why not via a command line (even if via the command line GUI)?

The “no-local-blocks” option is something nice for a test setting, but not for real-life use (at least not in my case).
Therefore I think it would better suit my needs if I can put this option “just for now” and not as a general setting, which you could potentially forget to disable.
And the GUI-based commandline seems ideal for that. Except if you have a better idea…

But as stated, if this is not possible, I can live with that. Big thanks for your help!

These are not the same optimizations. See the --no-local-blocks talk of “source files”.
This refers to the original location which was backed up, and not the --restore-path folder.
I don’t think there’s an option to force Duplicati to re-restore files that are already just fine.

That’s why the default is off – unless you’re testing – but I agree one could forget to flip it.

It should work on a GUI or true command line, but you still need to clear the restore folder.

I think we are missing eachother at some point.
When restoring, I choose a restore folder where the files are not present (so, not the location where the files are stored originally). By using the --no-local-block option I hope I don’t have to manually move the original files, in the original location, in order to “force” Duplicati to restore from the backup (cloud) location instead of just copying it from the original location.

So, more concretely:
Original location: MyOriginalFiles/file1.dat
Restore location: MyRestoredFiles/file.dat
Backup location: MyCloudFiles/file.dat

In this scenario, MyRestoredFiles/file.dat is not present, but MyOriginalFiles/file1.dat is.
My aim is to do a test restore (ensuring the quality and reliability of MyCloudFiles/file.dat) and do so without having to manually move MyOriginalFiles/file1.dat to some temporary location. For this I want to use --no-local-blocks.
It is by doing that via the command line that I get the “Restore completed without errors but no files were restored” message.

As stated, I don’t care putting the (global) GUI parameter for no-local-blocks (I’ll just put a warning for myself not to forget to unset the parameter) but I guess this is still an issue.

It’s never present. Duplicati doesn’t put file copies in the destination. The file format is hugely different. Regardless, let’s get back to original and restore locations.

Recall there have been separate usage issues. What happens if you don’t use --no-local-blocks?
The only way I can cause that error message is if I incorrectly specify files that I want to have restored.
Note that if you’re using a *, you need to put the specifier in quotes, otherwise your shell will expand it.

EDIT:

$ pwd
/tmp/MyRestoredFiles
$ rm file1.dat
$ mono /usr/lib/duplicati/Duplicati.CommandLine.exe restore file:///tmp/MyDestinationFiles/ "/tmp/MyOriginalFiles/*" --dbpath=/tmp/LGIJWQCDDQ.sqlite --no-encryption=true --restore-path=/tmp/MyRestoredFiles --console-log-level=information
Restore started at 3/1/2022 4:43:42 PM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (3 bytes)
Searching backup 0 (3/1/2022 9:31:24 PM) ...
Checking existing target files ...
  1 files need to be restored (5 bytes)
Scanning local files for needed data ...
1 remote files are required to restore
Backend event: Get - Started: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
  Downloading file (574 bytes) ...
Backend event: Get - Completed: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 1 (5 bytes) files to /tmp/MyRestoredFiles
Duration of restore: 00:00:03
***********************************************
Did we help save your files? If so, please support Duplicati with a donation. We suggest 10€ for private use and 100€ for commercial use.

https://www.duplicati.com/donate/
***********************************************
$ rm file1.dat
$ mono /usr/lib/duplicati/Duplicati.CommandLine.exe restore file:///tmp/MyDestinationFiles/ "/tmp/MyOriginalFiles/*" --dbpath=/tmp/LGIJWQCDDQ.sqlite --no-encryption=true --restore-path=/tmp/MyRestoredFiles --console-log-level=information --no-local-blocks
Restore started at 3/1/2022 4:44:06 PM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (3 bytes)
Searching backup 0 (3/1/2022 9:31:24 PM) ...
Checking existing target files ...
  1 files need to be restored (5 bytes)
1 remote files are required to restore
Backend event: Get - Started: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
  Downloading file (574 bytes) ...
Backend event: Get - Completed: duplicati-b6236107e507f483c96c3ce08fa15b18d.dblock.zip (574 bytes)
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 1 (5 bytes) files to /tmp/MyRestoredFiles
Duration of restore: 00:00:03
***********************************************
Did we help save your files? If so, please support Duplicati with a donation. We suggest 10€ for private use and 100€ for commercial use.

https://www.duplicati.com/donate/
***********************************************
$ 

Notice how adding --no-local-blocks stops Scanning local files for needed data ... step.
That’s what you should look at to see whether the option works. I think metadata is always downloads.
Note that with or without --no-local-blocks, I always get that file. Maybe you are asking differently?