Add metadata to backup files

Is there a good way to add metadata after compression??
I want to backup some files to let’s say google cloud, but I want to add some metadata about backup itself as a text or something else, there is no need to zip it, I want to create it on the fly. What would be the best way to do so.

Hello and welcome to the forum.

I’m not sure I follow exactly, but you could manually place a text file (for example) in the destination with the Duplicati data files. The text file would be ignored by Duplicati so it won’t cause any problems.

Or are you asking if Duplicati can do something like this automatically?

I want to have a text file containing the following information.
How many backups are in there and the dates respectively.
The backup_id (the nth backup successfully completed).
How long did the backups last.
And some more.
It doesn’t make sense to add files manually in the destination.

This can be determined by looking at the dlist files. The count of the files is how many backup versions you have, and the dates of each are part of the filename.

What do you mean by “backup_id”? Duplicati considers the most recent backup as version 0, the next oldest as version 1, and so on. The version number (backup_id) changes after each backup.

This is logged in the Duplicati web UI.

Agreed. Some of the information you request could be done with --run-script-after to generate a log file and then uploaded to your back end storage. You could even do a compare command in this script to log a full list of files that changed in this backup run.

Edit to add: I like this idea and may experiment a bit…

Actually, the version is exactly what I want, but where are they stored? how can I copy those in the destination backup as a means to recognize the backup and it’s state easily.

How about these? How can I put backup durations to the backup destination for each backup.
Again, the same question, where are they stored? (in order to do --run-script-after and copy those files)

I’m not sure I follow. The version number won’t mean much as the most recent backup is always version 0.

The stats are stored in the Duplicati database, they aren’t in files that can be copied directly to your back end storage. There might be a way to gather those stats in a --run-script-after script. I can try doing some testing.

If your goal is to retain a record of backups, have you considered using email reporting or http reporting? The web site duplicati-monitoring.com is excellent for the latter. I know it doesn’t put the reports in the same back end storage as your backup data though.

Here’s a previous thread on this topic:

The below might be handy. The result file contains some brief results including a run time.
Remote URL can feed Duplicati.CommandLine.BackendTool.exe, if destination requires it.

Sample result file stats. Messages follow later:

DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 0
ExaminedFiles: 1
OpenedFiles: 0
AddedFiles: 0
SizeOfModifiedFiles: 0
SizeOfAddedFiles: 0
SizeOfExaminedFiles: 14
SizeOfOpenedFiles: 0
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
ParsedResult: Success
Version: 2.0.5.111 (2.0.5.111_canary_2020-09-26)
EndTime: 12/20/2020 7:11:29 PM (1608509489)
BeginTime: 12/20/2020 7:11:25 PM (1608509485)
Duration: 00:00:03.3462766
MessagesActualLength: 11
WarningsActualLength: 0
ErrorsActualLength: 0

Another question is whether metadata should be per-backup-run in its own file (if so, what happens if the backup that it describes gets removed by retention policy) or a summary of all backups ever or still there.

EDIT:

For a summary without run time, here’s a find command run from GUI Commandline but it could easily be run from a true CLI or a script using Export As Command-line. See how retention policy thins old versions:

Running commandline entry
Finished!

            
Listing filesets:
0	: 2/8/2022 10:50:50 AM (7716 files, 12.20 GB)
1	: 2/8/2022 9:50:06 AM (7716 files, 12.20 GB)
2	: 2/8/2022 8:50:04 AM (7716 files, 12.20 GB)
3	: 2/8/2022 7:50:04 AM (7716 files, 12.20 GB)
4	: 2/8/2022 7:21:09 AM (7716 files, 12.20 GB)
5	: 2/7/2022 9:50:15 PM (7716 files, 12.20 GB)
6	: 2/7/2022 8:50:05 PM (7716 files, 12.20 GB)
7	: 2/7/2022 7:50:05 PM (7716 files, 12.20 GB)
8	: 2/7/2022 6:50:05 PM (7716 files, 12.20 GB)
9	: 2/7/2022 5:50:04 PM (7716 files, 12.20 GB)
10	: 2/7/2022 4:50:05 PM (7716 files, 12.20 GB)
11	: 2/7/2022 3:50:03 PM (7716 files, 12.20 GB)
12	: 2/7/2022 2:50:04 PM (7716 files, 12.20 GB)
13	: 2/7/2022 1:50:04 PM (7716 files, 12.20 GB)
14	: 2/7/2022 12:50:00 PM (7716 files, 12.20 GB)
15	: 2/7/2022 11:50:04 AM (7716 files, 12.20 GB)
16	: 2/7/2022 8:50:03 AM (7716 files, 12.20 GB)
17	: 2/6/2022 7:59:24 AM (7716 files, 12.20 GB)
18	: 2/5/2022 7:50:02 AM (7716 files, 12.19 GB)
19	: 2/4/2022 7:50:02 AM (7716 files, 12.19 GB)
20	: 2/3/2022 7:50:02 AM (7716 files, 12.19 GB)
21	: 2/2/2022 7:15:37 AM (7716 files, 12.19 GB)
22	: 1/25/2022 6:50:04 PM (7715 files, 12.14 GB)
23	: 1/18/2022 4:50:00 PM (7716 files, 12.36 GB)
24	: 1/11/2022 11:50:17 AM (7715 files, 12.36 GB)
25	: 12/13/2021 9:50:01 AM (6998 files, 12.31 GB)
26	: 11/12/2021 7:43:52 AM (6971 files, 12.29 GB)
27	: 10/6/2021 5:50:03 PM (706 files, 549.28 MB)
Return code: 0

A version number is just UI convenience identifier. Above is current backup state, but not total history.