I have backup set up to run every day 3:30 AM. Backup retention policy is set to “7D:1D,4W:1W,6M:1M”. But it looks like duplicatti deletes backup every other day
Could you please explain to me, why backup of the 10th of february was deleted?
Could you please explain to me, why backup of the 10th of february was deleted?
The retention can be a bit unintuitive, since it only looks at the time difference between two versions. Since your backups seem to run at different times (3am, 8am, 12pm), there might be only a 14h difference, which is less than one day and might cause a version to be deleted.
As you run the backup only once a day, you can keep every version for 7 days with 7D:U.
See also:
Thanks for the closer look. I was wrongly relying on
but there’s actually much more backing up going on
Duplicati.CommandLine.exe help retention-policy
--retention-policy (String): Reduce number of versions by deleting old
intermediate backups
Use this option to reduce the number of versions that are kept with
increasing version age by deleting most of the old backups. The expected
format is a comma separated list of colon separated time frame and
interval pairs. For example the value "7D:0s,3M:1D,10Y:2M" means "For 7
day keep all backups, for 3 months keep one backup every day, for 10
years one backup every 2nd month and delete every backup older than
this.". This option also supports using the specifier "U" to indicate an
unlimited time interval.
Here is today backup full log:
{
"DeletedFiles": 100,
"DeletedFolders": 0,
"ModifiedFiles": 1202,
"ExaminedFiles": 75547,
"OpenedFiles": 1342,
"AddedFiles": 140,
"SizeOfModifiedFiles": 244532556,
"SizeOfAddedFiles": 48488155,
"SizeOfExaminedFiles": 40219368782,
"SizeOfOpenedFiles": 318694429,
"NotProcessedFiles": 0,
"AddedFolders": 26,
"TooLargeFiles": 0,
"FilesWithError": 0,
"ModifiedFolders": 0,
"ModifiedSymlinks": 0,
"AddedSymlinks": 0,
"DeletedSymlinks": 0,
"PartialBackup": false,
"Dryrun": false,
"MainOperation": "Backup",
"CompactResults": {
"DeletedFileCount": 0,
"DownloadedFileCount": 0,
"UploadedFileCount": 0,
"DeletedFileSize": 0,
"DownloadedFileSize": 0,
"UploadedFileSize": 0,
"Dryrun": false,
"VacuumResults": null,
"MainOperation": "Compact",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-10T20:38:30.5696713Z",
"BeginTime": "2024-02-10T20:38:28.5602223Z",
"Duration": "00:00:02.0094490",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 11,
"BytesUploaded": 61351441,
"BytesDownloaded": 59234823,
"FilesUploaded": 5,
"FilesDownloaded": 3,
"FilesDeleted": 1,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 1352,
"KnownFileSize": 34789869928,
"LastBackupDate": "2024-02-11T03:30:00+07:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"VacuumResults": null,
"DeleteResults": {
"DeletedSetsActualLength": 1,
"DeletedSets": [
{
"Item1": 1,
"Item2": "2024-02-10T03:45:10+07:00"
}
],
"Dryrun": false,
"MainOperation": "Delete",
"CompactResults": {
"DeletedFileCount": 0,
"DownloadedFileCount": 0,
"UploadedFileCount": 0,
"DeletedFileSize": 0,
"DownloadedFileSize": 0,
"UploadedFileSize": 0,
"Dryrun": false,
"VacuumResults": null,
"MainOperation": "Compact",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-10T20:38:30.5696713Z",
"BeginTime": "2024-02-10T20:38:28.5602223Z",
"Duration": "00:00:02.0094490",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 11,
"BytesUploaded": 61351441,
"BytesDownloaded": 59234823,
"FilesUploaded": 5,
"FilesDownloaded": 3,
"FilesDeleted": 1,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 1352,
"KnownFileSize": 34789869928,
"LastBackupDate": "2024-02-11T03:30:00+07:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-10T20:38:30.5696713Z",
"BeginTime": "2024-02-10T20:38:24.7182458Z",
"Duration": "00:00:05.8514255",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 11,
"BytesUploaded": 61351441,
"BytesDownloaded": 59234823,
"FilesUploaded": 5,
"FilesDownloaded": 3,
"FilesDeleted": 1,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 1352,
"KnownFileSize": 34789869928,
"LastBackupDate": "2024-02-11T03:30:00+07:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"RepairResults": null,
"TestResults": {
"MainOperation": "Test",
"VerificationsActualLength": 3,
"Verifications": [
{
"Key": "duplicati-20240210T203000Z.dlist.zip.aes",
"Value": []
},
{
"Key": "duplicati-i000a0b8d0104476c977d3d3adfe96fe7.dindex.zip.aes",
"Value": []
},
{
"Key": "duplicati-bb07e6617964942c1a59922dfff0605f5.dblock.zip.aes",
"Value": []
}
],
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-10T20:39:04.7282778Z",
"BeginTime": "2024-02-10T20:38:33.8256309Z",
"Duration": "00:00:30.9026469",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null,
"BackendStatistics": {
"RemoteCalls": 11,
"BytesUploaded": 61351441,
"BytesDownloaded": 59234823,
"FilesUploaded": 5,
"FilesDownloaded": 3,
"FilesDeleted": 1,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 1352,
"KnownFileSize": 34789869928,
"LastBackupDate": "2024-02-11T03:30:00+07:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
},
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "2024-02-10T20:39:04.795402Z",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:09:04.7804691",
"MessagesActualLength": 31,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": [
"2024-02-11 03:30:00 +07 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started",
"2024-02-11 03:30:05 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()",
"2024-02-11 03:30:08 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (1,32 KB)",
"2024-02-11 03:35:52 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b7fb72f6a410e4af8b7fd6c24afe762b7.dblock.zip.aes (49,87 MB)",
"2024-02-11 03:35:58 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-be338dd0bac774a268c303508c277d073.dblock.zip.aes (1,93 MB)",
"2024-02-11 03:36:04 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-be338dd0bac774a268c303508c277d073.dblock.zip.aes (1,93 MB)",
"2024-02-11 03:36:05 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i040cc5d7d796498cb0e702bcb9b14936.dindex.zip.aes (12,39 KB)",
"2024-02-11 03:36:05 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i040cc5d7d796498cb0e702bcb9b14936.dindex.zip.aes (12,39 KB)",
"2024-02-11 03:38:05 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b7fb72f6a410e4af8b7fd6c24afe762b7.dblock.zip.aes (49,87 MB)",
"2024-02-11 03:38:06 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-i53c6cf9d39f3497ab398cdba096b8b72.dindex.zip.aes (191,90 KB)",
"2024-02-11 03:38:07 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-i53c6cf9d39f3497ab398cdba096b8b72.dindex.zip.aes (191,90 KB)",
"2024-02-11 03:38:07 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-20240210T203000Z.dlist.zip.aes (6,52 MB)",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-20240210T203000Z.dlist.zip.aes (6,52 MB)",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-StartCheck]: Start checking if backups can be removed",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-FramesAndIntervals]: Time frames and intervals pairs: 7.00:00:00 / 1.00:00:00, 28.00:00:00 / 7.00:00:00, 181.00:00:00 / 31.00:00:00",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupList]: Backups to consider: 10.02.2024 3:45:10, 09.02.2024 8:46:00, 07.02.2024 8:37:17, 05.02.2024 12:33:50, 03.02.2024 20:03:34, 26.01.2024 16:29:25",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-BackupsToDelete]: Backups outside of all time frames and thus getting deleted: ",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler:RetentionPolicy-AllBackupsToDelete]: All backups to delete: 10.02.2024 3:45:10",
"2024-02-11 03:38:24 +07 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteRemoteFileset]: Deleting 1 remote fileset(s) ...",
"2024-02-11 03:38:27 +07 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Delete - Started: duplicati-20240209T204510Z.dlist.zip.aes (6,51 MB)"
],
"Warnings": [],
"Errors": [],
"BackendStatistics": {
"RemoteCalls": 11,
"BytesUploaded": 61351441,
"BytesDownloaded": 59234823,
"FilesUploaded": 5,
"FilesDownloaded": 3,
"FilesDeleted": 1,
"FoldersCreated": 0,
"RetryAttempts": 0,
"UnknownFileSize": 0,
"UnknownFileCount": 0,
"KnownFileCount": 1352,
"KnownFileSize": 34789869928,
"LastBackupDate": "2024-02-11T03:30:00+07:00",
"BackupListCount": 6,
"TotalQuotaSpace": 0,
"FreeQuotaSpace": 0,
"AssignedQuotaSpace": -1,
"ReportedQuotaError": false,
"ReportedQuotaWarning": false,
"MainOperation": "Backup",
"ParsedResult": "Success",
"Version": "2.0.7.1 (2.0.7.1_beta_2023-05-25)",
"EndTime": "0001-01-01T00:00:00",
"BeginTime": "2024-02-10T20:30:00.0149329Z",
"Duration": "00:00:00",
"MessagesActualLength": 0,
"WarningsActualLength": 0,
"ErrorsActualLength": 0,
"Messages": null,
"Warnings": null,
"Errors": null
}
}
My PC is not turned on 24\7, as soon is started up, backup runs. Prevous backup started at 3:38AM the 10th of february, yet it was deleted. Does it have to start at EXACT SAME time every day to be considered one day apart?
3:38am on 10th to 3:30 on 11th is 23:52, so less than one day. That is why I suggested you keep all of the versions for a week, or reduce the time from one day to 12h. It is an unfortunate problem with the current retention system.
I got it. Thanks very much for reply.
I think it’s in backup order, so Backups to consider: 10.02.2024 3:45:10, 09.02.2024 8:46:00 means that the 10th is less than one day from the 9th, thus violates the 1 day spacing, so gets deleted. Latest backup is exempt from deletion, so 10th had no problem on the 10th but got deleted on the 11th.