Having trouble listing remote files

I’m trying to use the list (or is it find?) to list remote files.

I’ve tried the following: (some stuff obviously obfuscated here)

duplicati-cli list --ssh-fingerprint=“ssh-rsa 2048 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx” ssh://username:password@1.2.3.4/home/Backups/R1/OfficeMac/HD2

and the result is below.
Notice that (a) it’s telling me that some resource doesn’t exist and (b) regarding the supplied option --ssh-fingerprint that is apparently not supported, if I don’t include that, then it just tells me I need to include it!

Invalid type Microsoft.WindowsAzure.Storage.Blob.BlobEncryptionPolicy for instance field Microsoft.WindowsAzure.Storage.Blob.BlobRequestOptions:k__BackingField
Invalid type Microsoft.WindowsAzure.Storage.Queue.QueueEncryptionPolicy for instance field Microsoft.WindowsAzure.Storage.Queue.QueueRequestOptions:k__BackingField
Invalid type Microsoft.WindowsAzure.Storage.Table.TableEncryptionPolicy for instance field Microsoft.WindowsAzure.Storage.Table.TableRequestOptions:k__BackingField

Enter encryption passphrase:
No certificates found, you can install some with one of these commands:
cert-sync /etc/ssl/certs/ca-certificates.crt #for Debian based systems
cert-sync /etc/pki/tls/certs/ca-bundle.crt #for RedHat derivatives
Read more: Release Notes Mono 3.12.0 | Mono
The supplied option --ssh-fingerprint is not supported and will be ignored
No local database, accessing remote store
Listing remote folder …
Operation List with file attempt 1 of 5 failed with message: The folder /Applications/Duplicati.app/Contents/Resources/2048 does not exist => The folder /Applications/Duplicati.app/Contents/Resources/2048 does not exist
Listing remote folder …
Operation List with file attempt 2 of 5 failed with message: The folder /Applications/Duplicati.app/Contents/Resources/2048 does not exist => The folder /Applications/Duplicati.app/Contents/Resources/2048 does not exist

Any idea what I’m doing wrong?

Can you try to put the option at the end? After the ssh path.

If I do that I just get the message

Command not supported: 2048

It looks like your command is a little mixed up, because 2048 should be between quotes and therefore not parsed as a command.

The list command is an alias of find. As far as I know, list is deprecated, so replace it by find.

I guess this should work:
duplicati-cli find ssh://username:password@1.2.3.4/home/Backups/R1/OfficeMac/HD2 --ssh-fingerprint="ssh-rsa 2048 xx:xx:xx:xx:xx:xx:xx:xx:xx:xx"

The 2048 is part of that ssh fingerprint and so is in quotes. However, having done some more experiments, I’m noting the following:

After submitting in the order you suggested, I got back an error that the validation of the fingerprint failed. So I removed the fingerprint argument and then got back a message that I needed to add a fingerprint, and it told me to provide the exact same fingerprint that I was using!

It also suggested that I could use --ssh-accet-any-fingerprints instead so I tried that. This time I got the result

No local database, accessing remote store
Listing remote folder …

Listing filesets:
0 : 11/6/2017 12:00:00 PM
1 : 11/6/2017 8:00:00 AM
2 : 11/6/2017 5:09:41 AM
3 : 11/6/2017 12:00:00 AM
4 : 11/5/2017 8:00:00 PM
5 : 11/5/2017 4:00:00 PM
6 : 11/5/2017 12:00:00 PM
7 : 11/5/2017 8:00:00 AM
8 : 11/5/2017 5:13:24 AM
9 : 11/5/2017 1:00:00 AM
10 : 11/4/2017 9:00:00 PM
11 : 11/4/2017 5:00:00 PM
12 : 11/4/2017 1:30:55 PM
13 : 11/4/2017 12:54:29 PM
14 : 10/22/2017 5:13:54 PM
15 : 10/21/2017 4:59:08 PM
16 : 10/20/2017 6:02:53 PM
17 : 10/20/2017 3:43:42 AM
18 : 10/19/2017 6:46:35 PM
19 : 10/17/2017 10:57:04 PM
20 : 10/17/2017 5:15:12 AM
21 : 10/13/2017 2:10:25 PM

I suppose this is an improvement :slight_smile: but it still doesn’t look like a list of files!

What you’ve got is a list of versions (backup runs that included file changes).

When you say you’re trying to “list remote files” do you mean the archive files in your destination or the CONTENTS of those archive files (in other words, what’s been backed up)?

If it’s the list of backed up files you’re looking to get, then have you checked out the help associated with find that can be seen in Display Duplicati 2.0.2.1 beta command line help - #3 by JonMikelV?

Duplicati.CommandLine.exe help find

Usage: find <storage-URL> ["<filename>"] [<options>]

  Finds specific files in specific backups. If <filename> is specified, all
  occurrences of <filename> in the backup are listed. <filename> can contain
  * and ? as wildcards. File names in [brackets] are interpreted as regular
  expression. Latest backup is searched by default. If entire path is
  specified, all available versions of the file are listed. If no <filename>
  is specified, a list of all available backups is shown.

  --time=<time>
    Shows what the files looked like at a specific time. Absolute and relative
    times can be specified.
  --version=<int>
    Shows what the files looked like in a specific backup. If no version is
    specified the latest backup (version=0) will be used. If nothing is
    found, older backups will be searched automatically.
  --include=<string>
    Reduces the list of files in a backup to those that match the provided
    string. This is applied before the search is executed.
  --exclude=<string>
    Removes matching files from the list of files in a backup. This is applied
    before the search is executed.
  --all-versions=<boolean>
    Searches in all backup sets, instead of just searching the latest

Yes, if no filename is specified, a list of versions will be returned, as indicated in the help text. As wildcards are allowed, adding an asterisk (*) will return all files that are available in the latest backup version. Including the --ssh-accept-any-fingerprints you mentioned, the command would be:

duplicati-cli find ssh://username:password@1.2.3.4/home/Backups/R1/OfficeMac/HD2 * --ssh-accept-any-fingerprints

@JonMikelV @kees-z

Yes, I read the help and I tried using the asterisk after which I got the following:

Downloading file (101.67 MB)

and then after a long delay I got

Listing contents 0 (11/6/2017 12:00:00 PM):
/Volumes/HD2/src/boost_1_46_1/tools/build/v2/example/make/foo.py (68 bytes)
/Volumes/HD2/src/boost_1_46_1/tools/build/v2/test/dependency-test/foo.py (854 bytes)
/Volumes/HD2/src/boost_1_46_1/tools/build/v2/test/test2/foo.cpp (267 bytes)
/Volumes/HD2/src/boost_1_46_1/tools/build/v2/test/v1_testing/foo.cpp (319 bytes)
/Volumes/HD2/src/boost-build/test/test2/foo.cpp (257 bytes)
/Volumes/HD2/src/boost-build/test/v1_testing/foo.cpp (297 bytes)
/Volumes/HD2/src/Deskew/Projects/GPScript/TestScripts/hello.gpc (10.88 KB)
/Volumes/HD2/src/poppler-0.16.7/poppler/.libs/Array.o (34.71 KB)
/Volumes/HD2/src/poppler-0.16.7/poppler/Array.o (34.71 KB)
/Volumes/HD2/src/PyQt/sip-4.16.3/siplib/array.o (7.35 KB)
/Volumes/HD2/src/PyQt/Tests/foo.py (1.10 KB)
/Volumes/HD2/src/Python/foo.py (59 bytes)
/Volumes/HD2/src/Python/foo/untitled/foo.py (69 bytes)
/Volumes/HD2/src/Python/NamesAndAddresses/foo.py (159 bytes)
/Volumes/HD2/src/Python/PyCharm/untitled/foo.py (33 bytes)
/Volumes/HD2/src/PythonDebugTest/foo.py (393 bytes)
/Volumes/HD2/src/Qt5/5.3/Src/qttools/tests/auto/linguist/lupdate/testdata/good/merge_ordering/foo.cpp (2.64 KB)

The only file that has actually changed recently is

/Volumes/HD2/src/Deskew/Projects/GPScript/TestScripts/hello.gpc

All the others haven’t been touched in years.

Is this a consequence of the chunking mechanism?

If so, then this is a little problematic — if you have changed 5 (say) files since the last backup, then a backup report showing the most recently backed up files should really only show those 5 files

Some general thoughts ---- the mechanism for finding out what has been backed up (a very basic need, I would think) seems terribly complicated.

It seems to me that

a) in the web browser view, there should be a command that just lists all files backed up within some date range and/or perhaps since the last backup and

b) If you’re running the command line then why can’t you just refer to the job name instead of using an explicit storage-URL and all those extra options? After all that stuff is stored with the job already.

The find command returns all files in a backup set that matches the file filter (and * is any file, so everything that is included in the backup version is returned, changed or unchanged doesn’t matter).

To see what’s changed in a backup version (version 0), compared to the backup version before it (version 1) using the compare command:

duplicati-cli compare ssh://username:password@1.2.3.4/home/Backups/R1/OfficeMac/HD2 0 1 --ssh-accept-any-fingerprints --verbose

The --verbose options lists all files that were new, modified or deleted in the base version (0) compared to the other version (1). Without --verbose, only the first 10 new, modified and deleted files are listed.

You can compare any 2 versions, so if you want to know what happened to your files between backup version 5 and 17, replace 0 1 with 5 17 in the command above.

I agree. However, Duplicati started as a commandline only backup tool. In the current Web UI, a lot has been added the last couple of years. Useful features are added bit by bit to the Web UI. Feel free to add a feature request in the UX category with some ideas how it should look like in the UI.

It is there, click the Commandline link under the backup name and use the compare command.
Enter this in the arguments field:

0
1
--verbose

and click the Run "compare" command now button.

The commandline tool Duplicati.CommandLine.exe has no relation to the database that contains the configuration and backup jobs used by the Web UI. This is by design, so pointing straight to a backup job in the Web UI from the commandline is not possible.