Hello!
In order to “debug” my task I need to know what files have been included into each backup ‘version’. Although in the global settings I’ve turned “verbose = on”, I can’t find the list anywhere. What am I doing wrong?
MacOS, Duplicati 2.0.2.1_beta_2017-08-01
Thanks,
Sergey
Hi @Haron.
I had always thought that --verbose didn’t do anything, since enabling it in the job doesn’t include the list of files in the email notification.
I ended up writing my own notification script to compensate, using the cli compare option.
However, I just tested running cli jobs with and without compare option, and it does indeed include files.
No --verbose option (removing top section with my passwords and similar):
Backup started at 11/12/2017 9:20:20 PM
The supplied option --post_script_email is not supported and will be ignored
Checking remote backup …
Listing remote folder …
Scanning local files …
33 files need to be examined (73.06 MB)
0 files need to be examined (0 bytes)
No remote filesets were deleted
Checking remote backup …
Listing remote folder …
removing file listed as Temporary: duplicati-b26a4eb38fa6b4c929f6b86f5efb548ad.d
block.zip.aes
removing file listed as Temporary: duplicati-i3b6631d9db764074a2147df78ab40588.d
index.zip.aes
Verifying remote backup …
Remote backup verification completed
Downloading file (957 bytes) …
Downloading file (25.15 KB) …
Downloading file (36.25 MB) …
Duration of backup: 00:00:48
Remote files: 27
Remote size: 36.82 MB
Files added: 0
Files deleted: 0
Files changed: 0
Data uploaded: 0 bytes
Data downloaded: 36.27 MB
Backup completed successfully!
With the --verbose option, I see:
Backup started at 11/12/2017 9:24:26 PM
The supplied option --post_script_email is not supported and will be ignored
Checking remote backup …
Listing remote folder …
Scanning local files …
45 files need to be examined (73.12 MB)
Including source path: C:\BeSweetv1.4
Adding directory C:\BeSweetv1.4
Skipped checking file, because timestamp was not updated C:\BeSweetv1.4\azid.dll
…snip…
…snip…
Adding directory C:\BeSweetv1.4\Help - Copy
Checking file for changes C:\BeSweetv1.4\Help - Copy\2lame.txt, new: True, times
tamp changed: True, size changed: True, metadatachanged: True, 11/12/2011 6:41:1
2 PM vs 1/1/0001 12:00:00 AM
New file C:\BeSweetv1.4\Help - Copy\2lame.txt
Checking file for changes C:\BeSweetv1.4\Help - Copy\azid.txt, new: True, timest
amp changed: True, size changed: True, metadatachanged: True, 11/12/2011 6:41:12
PM vs 1/1/0001 12:00:00 AM
New file C:\BeSweetv1.4\Help - Copy\azid.txt
…skip…
0 files need to be examined (0 bytes)
Uploading file (4.58 KB) …
Uploading file (1.31 KB) …
Uploading file (4.20 KB) …
No remote filesets were deleted
Found 0 fully deletable volume(s)
Found 9 small volumes(s) with a total size of 542.47 KB
Found 0 volume(s) with a total of 0.00% wasted space (0 bytes of 36.91 MB)
Compacting not required
Checking remote backup …
Listing remote folder …
Verifying remote backup …
Remote backup verification completed
Downloading file (4.20 KB) …
Downloading file (1.31 KB) …
Downloading file (4.58 KB) …
DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 0
ExaminedFiles: 45
OpenedFiles: 12
AddedFiles: 12
SizeOfModifiedFiles: 0
SizeOfAddedFiles: 61402
SizeOfExaminedFiles: 76674477
SizeOfOpenedFiles: 61402
NotProcessedFiles: 0
AddedFolders: 1
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
ParsedResult: Warning
VerboseOutput: True
VerboseErrors: False
EndTime: 11/13/2017 2:24:41 AM
BeginTime: 11/13/2017 2:24:26 AM
Duration: 00:00:14.8824260
Messages: [
No remote filesets were deleted,
Compacting not required
As to why this info isn’t making it into the email, that is a good question, but the option certainly does create good output.
I still think using the compare option may be a better choice, since it shows the list of files added, modified and removed, whereas --verbose above shows every file looked at.
I’ve got a nice little script on Windows. If you are a scriptor who would like to Linuxify it, I’d be glad ot share.
Dave
You might want to try the --full-result setting as it spits out pretty much the same thing as --verbose but without the login info. 
As to why this info isn’t making it into the email, that is a good question
Indeed, let’s wait for the answer to THIS question
–verbose above shows every file looked at.
Well, than it’s not really useful as nowadays folders contain myriads of files.
I’ve got a nice little script on Windows. If you are a scriptor who would like to Linuxify it, I’d be glad ot share.
Regrettably I cannot do that.
Are you wanting to know every file in a version that would be recovered if you were to restore from that version or are you looking for a list of what files were added / updated / deleted in a specific version compared to another (such as the previous) version?
Correct. Actually, I’d like the backup process to be more transparent. Now Duplicati works as a black box. It would be a great chance to check that things goes well if the list of added/updated/deleted files was somehow accessible.
It is indeed available already - it’s just hiding on the --compare command.
It can be run via the “Commandline…” feature of the GUI as shown here:
Or from the command line as discussed here:
In both cases you may find --full-result gives you slightly cleaner results than --verbose.
Let us know if either of these works out for you.
It is indeed available already - it’s just hiding on the --compare command.
Well, not really
When I open that “commandline” URL for a particular backup job, there is no way to see the database path while it’s compulsory for “compare” command. Could you make it more smart and automatically include “–dbpath=/Users/haron/.config/Duplicati/XXX.sqlite” as the first line in the “Commandline arugments” box? Maybe for all commands (easier for implementation), maybe just for “compare” (easier for users).
After just a couple of seconds I can see A HUGE list of files which ends with some statistics:
Added files: 56
Deleted folders: 396
Deleted files: 7162
Modified folders: 18
Modified files: 53
However, I don’t see anything except approx 2000 lines of deleted files. It seems, all the other lines are lost, or maybe there is not a vertical scroller. I suppose. Moreover, the path of each file is so long that it can’t be visible without horizontal scrolling. Which is hardly possible taking into account height of the list.
To sum up, what I’m trying to say is that the output window is too high and too narrow at the same time, and has no vertical scroller. Maybe it’s a good idea to provide an option to export output as a text file. In such a case it would be absolutely easy to open in any viewer and analyse.
Yeah, in large delta jobs the change list can be longer than the web buffer.
I’ll see if I can provide something for you in a few hours that can output a text file or email for you. Otherwise a direct Duplicati.CommandLine.exe call may be the only way to go.
Unfortunately, this command line function doesn’t work:
Input command: compare
Input arguments:
file:///Volumes/auto/Hermes/?auth-username=backup&auth-password=2A%Try
Input options:
dbpath: /Users/haron/.config/Duplicati/XXXXX.sqlite
all-versions: false
backup-test-samples: 0
thread-priority: idle
verbose: true
disable-autocreate-folder: true
backup-name: MBP
encryption-module: aes
compression-module: zip
dblock-size: 100MB
passphrase: XXXXXX
keep-time: 1Y
prefix: MBP
exclude-files-attributes: temporary,hidden
disable-module: console-password-input
Mono.Data.Sqlite.SqliteException (0x80004005): SQLite error
near ""."": syntax error
at Mono.Data.Sqlite.SQLite3.Prepare (Mono.Data.Sqlite.SqliteConnection cnn, System.String strSql, Mono.Data.Sqlite.SqliteStatement previous, System.UInt32 timeoutMS, System.String& strRemain) [0x00236] in <7797d8bb38254a72bf06be6a3048939d>:0
at Mono.Data.Sqlite.SqliteCommand.BuildNextCommand () [0x00059] in <7797d8bb38254a72bf06be6a3048939d>:0
Return code: 100
Welcome to the forum! I edited your post to improve the formating. (Just added ~~~ before and after the output you pasted, please see here for details).
Can you include the command that produced those results?
For me, this command works:
Duplicati.CommandLine.exe" compare "file://E:\_Duplicati-Block -Test" --dbpath="E:\_Duplicati-SQLite\NJQGAWWNCA.sqlite" 0 1 --full-result
Including the --dbpath parameter speeds things up as it doesn’t have to pull data from the destination, plus I believe it avoids having to include a password (or the disable password parameter).
No command, just “Commandline” page:
I’ve gotten that "error hear ‘.’ error myself and it was always due to something wrong with my parameters.
I’d recommend:
--full-result is on its own line (it’s not indicated, but each command line argument needs to be its own line)It helped, thanks!
Now waiting for a bugfix to be able to observe the whole (long) output…
Too many changes to fit in the browser scroller? You might want to shell / CLI call as shown in this post with a redirect to an output file (add >>C:\DuplicatiOut.txt to the end of line).
While I hope to see output routing added to the GUI based command line, I don’t believe it’s on the short list for the next release. 
As I can see, there is no file sizes in the log. How can I check what files are eating space?
Any ideas how to track (and reduce) backup size?
Not in a simple way, no. You could probably dig into the .sqlite file and get file sizes and the like from that but you’d basically be writing your own SQL to do it.
Reducing backup size pretty much comes down to some combination of:
I’d suggest you start with looking at what’s actually being backed up and make sure you’re not grabbing junk you don’t need. I’m not sure what all Macs keep around but on Windows it’s easy to accidentally be backing up temp, paged memory, hibernation, and recycle bin files).
You could set the “Advanced options” --zip-compression-level higher (such as to 9 for maximum compression). Not that this is found in the “Zip compression” section of the “Add advanced option” select list.
You might also want to look at the --retention-policy feature (although it was added to a canary version after your beta, so you’d have to update to that version or wait for the feature to come to your upgrade path).