Backup detailed summary / file list

Hello,

It would be great to see list of files added / modified / deleted in the backup.
after backup I can see just the summary (number of files and their total size):
Examined 18605 (5.43 GB)
Opened 24 (10.11 MB)
Added 22 (9.77 MB)
Modified 2 (350.01 KB)
Deleted 4

It would be awesome to see filenames (with full paths) for example after clicking on those summaries.
or even better - some “DETAILS” button, that would show complete list of SOURCE filenames with full paths.

Welcome to the forum @Artur2048

Although a button would be nicer, you can get a list of full source paths in a backup using list command either from Commandline in the web UI, or from your OS command line where sending it to file is simpler. The FIND command describes this. Basically, you use the wildcard * if you want all the pathnames listed.

Simple sample output:

Listing contents 0 (6/25/2020 5:03:37 PM):
C:\backup source\short.txt (7 bytes)
Return code: 0

Although a button would be nicer, The COMPARE command can show you the added / modified / deleted.

Simple sample output:

image

Listing changes
  1: 6/27/2020 8:10:01 AM
  0: 6/27/2020 8:10:50 AM

Size of backup 1: -1 bytes

  1 added entries:
  + C:\tmp\compare\file3.txt

  2 modified entries:
  ~ C:\tmp\compare\
  ~ C:\tmp\compare\file2.txt

1 deleted entries:
  - C:\tmp\compare\file1.txt


  Added files:       1
  Deleted files:     1
  Modified folders:  1
  Modified files:    1
Size of backup 0: 7 bytes
Return code: 0

So there’s your added / modified / deleted. I don’t think Examined and Opened are logged directly, because backup records are on what was backed up, not what wasn’t. With no filtering, Examined seems like it will be close to the list of all files mentioned earlier. Opened is a check for changes. Goes Modified if it was…

compare command does become harder to use for something other than the two most recent backups, as you need to tell it what you want, and might need to read the restore dropdown to find the version numbers:

image

Above are available right now. Maybe they’ll help you some until a nice user inteface enhancement is done.

Thank you so much!
Compare command is exactly what i was looking for. please please please make it a nice button on backup summary :slight_smile:

There is one issue however… if the list of files is long, it’s not displayed in full…
instead Duplicati would show just few first files from each section with comment “…and X more”.

is there any way to display the full list regardless of the size?

example of my output is as follows:

Size of backup 1: 5.43 GB

14 added entries:

  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
  • zzzzzzzzzzzzzzzzz
    … and 3 more

15 modified entries:
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
~ xxxxxxxxxxxxxxxxx
… and 4 more

3 deleted entries:

  • ddddddddddddddddd
  • ddddddddddddddddd
  • ddddddddddddddddd

Added files: 14
Deleted files: 3
Modified folders: 12
Modified files: 3
Size of backup 0: 5.44 GB
Return code: 0

Possibly full-result (not tested):

Use this option to increase the amount of output generated as the result of the operation, including all filenames.

Thank you very much again.
“full-result” together with compare command works perfectly, and does exactly what i was looking for.

I already made shortcut on my desktop to check what happened in last backup :slight_smile:

it would be really awesome to have for example “add/modified/deleted” sections expandable in backup summary. We have that logic already existing for WARNINGS (list of files that were not included in backup)

Or maybe just make one LOG showing files:

  • warnings
  • added
  • modified
  • deleted

Just some “user friendly” way of accessing that kind of information

one more thing… is it also possible to get last backup status, the list of errors, and the list of warnings using command line interface?

i’m trying to get some kind of backup summary, that would show:

  • status (was it ovarall success?)
  • list of errors
  • list of warnings (and source filenames, which were not included for some reason)
  • list of files added, modified, and deleted
  • backup size, timestamp, and the time it took to perform
  • etc…

All of above except the list of files added/modified/deleted can be found on the Duplicati console in the log.

i’m just trying to have it in one place somehow.

Using Duplicati from the Command Line doesn’t show such a thing that I can see, and neither does help.

Possibly some of this is due to the history. Command line came first, then web UI, which added features.
Raw info for warnings are in the per-job database, but errors are in the all-jobs database which GUI adds. You can see one unfortunate result of this in an early error case – backup logs view doesn’t have that log.

One good way to capture information, including some that’s nowhere else, is to use the reporting options. These can do things like send you email, or provide information via an HTTP web request to a web server.

Your own tools, or third-party tools such as Duplicati Monitoring or dupReport can build on the basic feeds. Both have support threads in this forum. They’re likely less backlogged on feature requests than Duplicati.

Thank you very much for all the answers and suggestions.

I’m running Duplicati just to backup my laptop and test the solution at the same time :slight_smile:
…so i would like to have one “unified” simple tool that does it all :slight_smile:
Duplicati is great solution. it’s currently in Beta version.
…i hope when it becomes stable-release user interface gets a bit more user friendly :slight_smile:

I’m looking for a similar solution - list of all files backed up during the last task. Using COMPARE with --full-result works, but there is one major issue:

My backup sometimes flags certain files as “modified” based on metadata, but when it does a full read/hash/check of the file, it’s detected as “metadata-only changed” and not backed up. This file is (correctly) not backed up by Duplicati, but still shows up in the list of modified files generated by compare. (The summary shows ModifiedFiles: 1 and SizeOfModifiedFiles: 0).

Is there a way to get a list of files actually backed up during the last task?

yes it is, the metadata is part of the file, and it’s correct it is listed as modified.

Appreciate the quick reply, and yes I agree that Duplicati is correctly listing it as modified given that the timestamp has changed by ~400ms. My question is whether there is a way to obtain a list of files that were actually backed up - i.e. files that were actually transmitted to the backup destination and are part of the SizeOfModifiedFiles data.

I don’t see an obvious way to do that with default Duplicati functions. Could probably be hacked by processing logs or even querying the database but complicated.

Possible query that for now needs manual help based on look into Fileset table to find highest two ID:

WITH
New AS (
	SELECT Path,BlocksetID FROM FilesetEntry JOIN File ON FilesetEntry.FileID = File.ID WHERE FilesetID = 633
),
Old AS (
	SELECT Path,BlocksetID FROM FilesetEntry JOIN File ON FilesetEntry.FileID = File.ID WHERE FilesetID = 632
)
SELECT New.Path FROM New JOIN Old WHERE New.Path = Old.Path AND New.BlocksetID <> Old.BlocksetID;

This is just automating (for all common files) the manual database study that I suggested be done for
Files incorrectly flagged as modified which I think hit the actual question: what files had data change?
We’re getting extra metadata changes mixed in. They cause small uploads (metadata is pretty small).
More annoyingly, they cause file scans and it’s not clear what gets found. Above SQL can help a little.

1 Like

I’m using the following commands:

@cd "C:\Program Files\Duplicati 2\"
@Duplicati.CommandLine.exe compare box://Duplicati/backup_dir?authid=xxxxxxxxxx --passphrase=yyyyy --full-result=true
pause

It shows the list of modified files, added files and deleted files in last backup

The challenge being discussed is that Duplicati considers a file to be modified if its data changes or its metadata such as security attributes and creation time changes. Please read at the link for the context.

I think the request was to be able to list files where data changed, ignoring the bogus metadata change.

You can test that compare command considers timestamp changes to be a modification – re-save a file.

Thanks for the idea, but I’m not sure it suits this somewhat odd situation of unwanted metadata change.

for sure it’s not a common need. Adding it to Duplicati would not be very hard, but no one has felt the urge.