Can't figure out how to restore all files from a folder via the command line

Support
1

Hello,

I’m using the following command line on Manjaro Linux to create a backup without history for 2 not so important folders:

duplicati-cli backup --backup-name="No retention" --dbpath=/pat/to/database.sqlite --encryption-module=aes --compression-module=zip --dblock-size=50mb --passphrase="XXXX" --disable-module=console-password-input --asynchronous-concurrent-upload-limit=10 --keep-versions=1 "b2://bucket_name/?auth-username=user&auth-password=password" /first/folder /some/other/folder

This works just fine as I see it processing quite a few files and the bucket gets filled with encrypted files.

Through my own stupidity, I deleted an entire sub folder two levels inside /first/folder and so I thought this would be the perfect time to try the restore command and did it like so:

duplicati-cli restore --backup-name="No retention" --dbpath=/pat/to/database.sqlite --encryption-module=aes --compression-module=zip --dblock-size=50mb --passphrase="XXXX" --disable-module=console-password-input --asynchronous-concurrent-upload-limit=10 --keep-versions=1 "b2://bucket_name/?auth-username=user&auth-password=password" /first/folder/down/subfolder

While this does not error out, it does not restore anything:

Restore started at 24/07/2024 11:29:11
Checking remote backup ...
  Listing remote folder ...
Checking existing target files ...
Verifying restored files ...
Restore completed without errors but no files were restored
Restored 0 (0 octets) files to original path
Duration of restore: 00:00:55

I found this quite strange so I tried to use find, like so:

duplicati-cli find --backup-name="No retention" --dbpath=/pat/to/database.sqlite --encryption-module=aes --compression-module=zip --dblock-size=50mb --passphrase="XXXX" --disable-module=console-password-input --asynchronous-concurrent-upload-limit=10 --keep-versions=1 "b2://bucket_name/?auth-username=user&auth-password=password"

This gave me this:

Listing filesets:
0       : 19/07/2024 05:23:07 (368582 files, 684,645 GB)

which is on par with the latest run for this backup.

If I give /first/folder/down/subfolder as an additional argument to find, I get this:

No files matching, looking in all versions
No files matched expression

And as soon as I try using search expressions, the find command consumes CPU for a long time and appears to never return.
Here are a few that I tried:

'[\/first\/folder\/.+]'
'/first/folder/*'
"/first/folder/*"
/first/folder/*

Clearly, I must have missed something but I can’t figure out what it could be.

Thanks for any help

2

This one restores only the folder /first/folder/down/subfolder which is why you get the warning that no files are restored. If you add an asterisk to the path, you will get all, but the shell will expand it locally, so you need to escape/guard it. Something like this should work:

duplicati-cli restore --dbpath=/pat/to/database.sqlite --passphrase="XXXX" "b2://bucket_name/?auth-username=user&auth-password=password" '/first/folder/down/subfolder/*'

Note the single quotes which prevents the shell expansion. Alternative is something like "/first/folder/down/subfolder/\*".

Not sure what happens with the list method, but there was a case-sensitive bug found in v2.0.9.1 that only manifested itself on Linux, so that might be it.

3

Aha! That was it!
Thanks for the tip, it worked perfectly after that.

Nice to know that, I’ll see if that changes in the next version

4

I’m glad it helped. I’ve tripped over this somewhat surprising behavior before, but couldn’t reproduce it until today. More questions and Windows results on some different scenarios:

I’m not sure that’s the case. On Windows it won’t restore last folder for me after I deleted it.

C:\>"C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\Duplicati.CommandLine.exe" restore "file://C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\RUN\test 4" "C:\backup source\folder" --dbpath="C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\RUN\RHRJDWVWXA.sqlite" --no-encryption=true --console-log-level=verbose
Restore started at 7/25/2024 9:23:23 AM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (3 bytes)
Searching backup 0 (7/25/2024 1:05:30 PM) ...
Restore list contains 0 blocks with a total size of 0 bytes
Checking existing target files ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restore completed without errors but no files were restored
Restored 0 (0 bytes) files to original path
Duration of restore: 00:00:01

and the folder is in fact not restored. It restores if I put a backslash at the end, which is what I was trying yesterday. I was wondering why what I thought would fail was working for me. On Windows the trailing backslash must be doubled if the whole path is double quoted, otherwise it makes the closing double quote into a literal double quote. System using forward slash don’t have this oddity.

C:\>"C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\Duplicati.CommandLine.exe" restore "file://C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\RUN\test 4" "C:\backup source\folder\\" --dbpath="C:\Duplicati\dupl
icati-2.0.8.1_beta_2024-05-07\RUN\RHRJDWVWXA.sqlite" --no-encryption=true --console-log-level=verbose
Restore started at 7/25/2024 9:24:21 AM
The operation Restore has started
Checking remote backup ...
Backend event: List - Started:  ()
  Listing remote folder ...
Backend event: List - Completed:  (3 bytes)
Searching backup 0 (7/25/2024 1:05:30 PM) ...
Restore list contains 1 blocks with a total size of 139 bytes
Creating folder: C:\backup source\folder\
Checking existing target files ...
  0 files need to be restored (0 bytes)
1 remote files are required to restore
Backend event: Get - Started: duplicati-b847d0eba70964a808d70121b0c2c7915.dblock.zip (494 bytes)
  Downloading file duplicati-b847d0eba70964a808d70121b0c2c7915.dblock.zip (494 bytes) ...
Backend event: Get - Completed: duplicati-b847d0eba70964a808d70121b0c2c7915.dblock.zip (494 bytes)
Recording metadata from remote data: C:\backup source\folder\
Patching metadata with remote data: C:\backup source\folder\
Verifying restored files ...
Restore completed without errors but no files were restored
Restored 0 (0 bytes) files to original path
Duration of restore: 00:00:02

The asterisk technique works too, but instead of slash star end it seems one can just just slash.

Regarding backslash-escaping star in double quotes, I don’t think it’s necessary in bash or zsh.