Keep-time confusion

#1

I’ve read through a ton of the retention policy threads but I’m still confused about something. My goal is to keep one copy of every file and file revision (sort of a backup/archive hybrid) without having 500000000 backup versions. :slight_smile: So, if I set my keep-time to, say, 100 years (if that’s even possible), and my keep-versions to 1, will that achieve what I what, i.e. 1 instance of every file and file revision with a minimal number of backup versions? Thanks.

#2

The default is to keep all versions without limit, so I’m not sure why you’d have to get into --keep-time at all.

Not unless I misread you. That’s telling Duplicati to keep only the latest version (which might have old files).

A version is a view of the files at a point in time, but only changes are uploaded. Unchanged file data is just referenced as it was originally, so the storage space of the new versions is less than it would otherwise be. The database in 2.0.4.5 beta stores paths of all files of every version, but current canary versions use less.

If the goal is to keep a copy of every file forever, you’ll get a lot of versions, but few actually changed files in each new one (depending on how often you run backups – default is no changes means no new versions).

Personally I get nervous from hearing “archive” and “100 years”. Duplicati is still new and less than perfect. Anything meant for serious archival storage should be highly reliable, and perhaps even aimed at archiving. Duplicati is for backups, not for (for example) meeting archival needs in cases where originals are deleted.

#3

Well but the difference between backup and archive, for a home user at least, is pretty thin I think because the point of backup software is to retain copies of files in the event of hard drive failure or user deletion. Given that storage is so cheap, a “save everything forever” model seems to make sense regardless of whether Duplicati is being used for personal or professional needs. For example: a photo album with thousands of jpgs where you want to save all the pictures in your backup forever but only keep a smaller crop of current ones on your laptop which you can delete knowing they’ll be somewhere in the backup.

I get that Duplicati’s beta, though, and that it works in backup versions rather than file versions so its design is that you have to remember a specific backup date as finding all versions of a particular file, while doable, is a bit clunky at the moment.

#4

I consider these to be the sort of short-term protection of potentially old files that’s not archive-then-delete, therefore I personally think this is a relatively low-risk use case of software that sometimes has problems.

This use case is the one that worries me. Even though Duplicati has about four levels of restore available, they get increasingly difficult should a restore problem arise. Generally what seems to happen (per forum) isn’t a surprise restore failure, but an internal self-check going off alerting the user to the potential problem. Sometimes efforts to fix things aren’t enough or make matters worse. Sometimes it’s just easier to restart. That’s tolerable (though not really wanted) in the backup case, but not in the archive-and-delete just above.

There have been forum requests for various use cases of archive-and-delete, but historically the answer is that Duplicati is for backup, not archive. A harder reality is that issues like Duplicati reliability deserve work, and feature enhancements may wait. It’s not the same skillset, though. Some can do features but not core. Some users like that, but some gripe that core issues were neglected. My view is people do what they can.

Regarding finding all versions of a file, very clunky is a search with restores. The FIND command is better:

Listing files and versions:
C:\emptytests\length1.txt
0 : 3/4/2019 7:05:18 PM 1 bytes
1 : 3/4/2019 7:02:13 PM 1 bytes

There have been requests in the forum (which I can’t find quickly) of putting some sort of per-file date on the Restore files UI. It might have been just a date, or more radical such as a dropdown for every file.